#875: TAG review for web app `scope_extensions`

Visit on Github.

Opened Jul 20, 2023

Hola TAG!

I'm requesting a TAG review of scope-extensions.

This document describes a new scope_extensions manifest member that enables web apps to extend their scope to other origins. This allows sites that control multiple subdomains and top level domains to behave as one contiguous web app and also enables web apps to capture user navigations to sites they are affiliated with.

  • Explainer¹ (minimally containing user needs and example code): Scope Extensions explainer
  • Specification URL: N/A
  • Tests: N/A
  • User research: N/A
  • Security and Privacy self-review²: url
  • GitHub repo (if you prefer feedback filed there): url
  • Primary contacts (and their relationship to the specification):
  • Organization(s)/project(s) driving the specification: Microsoft (Edge)
  • Key pieces of existing multi-stakeholder review or discussion of this specification: N/A
  • External status/issue trackers for this specification (publicly visible, e.g. Chrome Status): ChromeStatus entry

Further details:

  • [ X ] I have reviewed the TAG's Web Platform Design Principles
  • Relevant time constraints or deadlines: [please provide]
  • The group where the work on this specification is currently being done: WICG
  • The group where standardization of this work is intended to be done (if current group is a community group or other incubation venue): WICG
  • Major unresolved issues with or opposition to this specification: N/A
  • This work is being funded by: Microsoft

You should also know that...

[please tell us anything you think is relevant to this review]

We'd prefer the TAG provide feedback as (please delete all but the desired option):

💬 leave review feedback as a comment in this issue and @-notify [luhuangmsft and diekus]

Discussions

Discussed Sep 1, 2023 (See Github)

Amy: what actual additional power does this grant? What happens if a site lies about their urls?

Peter: the manifests scopes down the origin.. and this expands it... doesn't say if it grants access to data of the other origins or not. Doesn't say one way or the other. Does allow you to capture links. You download a malicious app, install it, it declares it's scoped to your bank, it has a link to your bank, displays a url that looks like your bank, and it fishes all your credentials from a login page. It does say your bank has to have an intercept links authorization for that to happen

Dan: where is that configured? THe bank?

Peter: presume that would be in a manifest file for your bank

Amy: as long as that's opt-in..

Peter: maybe not for a random bank, but amazon might use this and say yes intercept links because they're using it within their own domains, so a malicious app could then use it to phish your amazon credentials. It says it's allowing anybody to intercept links, not a specific list of other origins to intercept links. If it's a bidirectional association and both sides have to have installed apps with that association in the manifest, and all it does is allow link capturing and prevent the ui from saying you're going to another site, and doesn't expose any origin information, I could live with this. But there's a bunch of ifs.

<blockquote> Hi @diekus – Briefly, we're very concerned about the way that this proposal changes the same-origin model, which is a fundamental part of the security aparatus of the web. Hence we think we need to tread very carefully. We think the explainer should be very explicit about what the expanded scope does and does not allow access to. We'd also like to see some specific use cases and discussion of abuse cases (and how those abuse cases are mitigated). E.g. if you are tricked into visiting or downloading a malicious app that is spoofing your bank, and it includes your bank's origin in its scope_extensions field, are there additional exploits that the malicious party could exploit (e.g. obtaining credentials or capturing links)? Are there any implications for access to local storage from different origins? </blockquote>
Comment by @alancutter Sep 1, 2023 (See Github)

The link to the security review needs updating: https://github.com/WICG/manifest-incubations/blob/gh-pages/scope_extensions-security-privacy-questionnaire.md

Comment by @torgo Sep 7, 2023 (See Github)

Hi @diekus – Thanks for sending us this. Briefly, we're concerned about the way that this proposal changes the same-origin model, which is a fundamental part of the security apparatus of the web. Hence we think we need to tread very carefully. We think the explainer should be very explicit about what the expanded scope does and does not allow access to. We'd also like to see some specific use cases and discussion of abuse cases (and how those abuse cases are mitigated). E.g. if you are tricked into visiting or downloading a malicious app that is spoofing your bank, and it includes your bank's origin in its scope_extensions field, are there additional exploits that the malicious party could exploit (e.g. obtaining credentials or capturing links)? Are there any implications for access to local storage from different origins?

Comment by @LuHuangMSFT Sep 28, 2023 (See Github)

@torgo Discussed with @diekus and below are our thoughts. I'll organize the important parts and add to the explainer. Thanks for your feedback.

  • Storage and permissions models are unaffected by this proposal and should remain so.
  • In Chromium, there are 2 behaviors we are working on in the near term that will be able to influenced by scope_extensions. These are essentially UX treatments.
    • Web app windows will not display out-of-scope UI when navigating within extended scope
    • Links to extended scope can cause the app window to launch

To prevent spoofing attacks, the implementation in Chromium will flash the web origin of the content in the window title bar after every top level navigation. The origin information will also be visible in the app's main menu.

If the user is tricked into visiting or installing a malicious app that is spoofing my bank

  • they can already do this without scope_extensions
  • if the malicious app includes my bank's origin in scope_extensions but my bank does not validate the association, my bank's content will appear with out-of-scope UI, and won't share any storage or permissions regardless. This is no different that if the malicious app simple redirected to my bank's site.

To use scope_extensions, the owner of the app should either also directly own/control the listed origins in scope_extensions or monitor them closely if working by agreement with parties that own them.

Browser security tools such as Microsoft Defender SmartScreen should still identify unsafe origins that are navigated to from the app window.

Discussed Oct 1, 2023 (See Github)

Dan: ...same origin.. malicious apps.. exploits? Manifest file that in some ways mirrors the functions of FPS.. but on a web app basis.. we want to be clear about the scope

Yves: basically that. Also wondering why they don't do cryptographic verification of the other origin they want to add in like it's done for some other places. Like what we ask from miniapps. We want to ensure that the other origin that we want to cover allows being embedded in this web app. Nothing there that talks about that.

Dan: given the response could you leave that feedback?

Yves: yes

Dan: it's a good response addressing the issues that we've raised. I don't think we should rely on something like safe browsing modes that are to protect users

Yves: thinking of things like universal links for ios and the equivalent on android which is always the canonical example I use

punts to next week

Discussed Oct 1, 2023 (See Github)

Dan: they responded..

Yves: the fact that app links are verified by apple to be able to reach the store, and we don't have that here. There is a verification step that you don't see in the end product but that happened by going to the store. So it has to be replaced in another way in the case of a regular web app [leaves feedback]. No response on webkit and mozilla standards positions.

Comment by @ylafon Oct 10, 2023 (See Github)

Hi, we discussed the issue during our breakout today. Could there be cryptographic proof that the added origins are agreeing to being embedded in that web app. Pretty much like Universal Link or App Link?

Comment by @LuHuangMSFT Oct 10, 2023 (See Github)

The apple-app-site-association file used by Universal Links references apps by an appID string of the format <Application Identifier Prefix>.<Bundle Identifier>. [1] I don't see usage of a cryptographic hash. Please correct me here if I'm missing something.

assetlinks.json, used by Android App Links, refers to apps by an app id and SHA256 fingerprints of the app's signing certificate. [2]

Use of a unique app id [3] should be sufficient evidence that the added origins are agreeing to being embedded in that uniquely identified web app. In the scenario where the app is signed or delivered as an immutable package, use of a cryptographic hash would be useful to further specify that the association is only valid when the app is unchanged. Being able to specify that the app remain unchanged doesn't seem like a useful feature for web apps with frequently changing content served through the web.

One scenario we should consider: if the web app is taken over by another party which does not have access to the original signing certificate, they would be unable to modify the app and produce cryptographic evidence matching the original - thus the origin association would become invalid.

The dominant method of delivery of web apps is over the web and managed by a browser without signing or packaging/bundling. Referencing web apps by unique app id is an acceptable solution that doesn't significantly complicate the steps developers need to take to set up the association.

To mitigate app takeover issues (where app ownership changes), we recommend that the web app and associated origins are owned and controlled by the same entity. Failing that, both the app and associated origins are advised to monitor ownership and condition of their counterparty.

[1] https://developer.apple.com/documentation/xcode/supporting-associated-domains [2] https://developer.android.com/training/app-links/verify-android-applinks#web-assoc [3] https://github.com/philloooo/pwa-unique-id/blob/main/explainer.md#requirements

Comment by @alancutter Oct 11, 2023 (See Github)

Could there be cryptographic proof that the added origins are agreeing to being embedded in that web app.

Is it sufficient that the added origin explicitly grants the web app permission in a file hosted on the origin served over HTTPS?

Comment by @ylafon Oct 17, 2023 (See Github)

In the case of apps, the verification of app links is done by the store owner, in the case of a web app, you need to have verification done in another way. This is similar to the origin issue in MiniApps Having something hosted on the origin server might work, but it would require network access.

To mitigate app takeover issues (where app ownership changes), we recommend that the web app and associated origins are owned and controlled by the same entity.

controlled in legal term (in which jurisdiction?) or in technical terms (as same hosting)

Comment by @LuHuangMSFT Oct 17, 2023 (See Github)

Controlled in practical terms - as it requires some level of control over the domain to be able to add /.well-known/ configuration. What's an appropriate way to phrase that in spec language?

Discussed Nov 1, 2023 (See Github)

Yves: the fact that app links are using something similar to what is done for letsencrypt - requesting a hash to be put somewhere... different than putting something into .well-known ... verification stage. If you don't have that - the verification - then I think it's dangerous.

Dan: We need to feed back that they need some kind of cryptographic proof...

Yves: it can be many things - cryptographic verification - in that case it's about the manifest... in that case you need to ensure that the verification is done off-line. Or if it's online then you need something similar to what exists in the acme protocol... It's prettty similar to what we have in miniapps...

Yves: their answer is more "how can we do that?"

yves to leave feedback in that regard

Yves: you can do it but only in this case - and look at what acme has done for online use case, look at what miniapp is working on for off-line case.

Discussed Nov 1, 2023 (See Github)

no response yet

Discussed Nov 1, 2023 (See Github)

Yves: don't know if they want to circumvent cross domain comms ...

Comment by @ylafon Nov 20, 2023 (See Github)

Well, the site has to agree that a web app can impersonate its content, so it is more than putting one file and be done (as other web apps could rely on its presence and do the same), there need to be something along the line of what is done in the ACME protocol, or else, in the case of offline web apps, a way to check this offline.

Discussed Dec 1, 2023 (See Github)

Yves: they want to have the webapp serve things on behalf of other origins. Wondering what is their understanding in ensuring things have to be done through https or with something that has the same properties in order to have powerful features, that might be an issue

Dan: Looks like we haven't received a reply from the proposers. Nudge for an update? There shouldn't be an issue with IWA here

Yves: scope extension to a regular webapp

Dan leaves comment

Comment by @alancutter Dec 1, 2023 (See Github)

as other web apps could rely on its presence

The site identifies a particular web app to associate with via the web app's manifest id, other web apps will be excluded by this.

in the case of offline web apps, a way to check this offline.

Does "offline" mean offline at all stages even during installation? A fully offline web app should probably be built as an IWA rather than use any HTTPS.

Comment by @torgo Dec 19, 2023 (See Github)

Hi @LuHuangMSFT can you update us and let us know your thoughts on @ylafon's question above?

Also – I don't think you want to draw a dependency to IWA from this proposal - can you confirm? Regardless of anything else it feels like this should be operating in secure contexts.

Discussed Jan 1, 2024 (See Github)

Yves: i view this as having the same security problem - trusting the origin - as miniapp...

Sangwhan: leaves comment

Comment by @LuHuangMSFT Jan 23, 2024 (See Github)

@ylafon The site identifies the web app by its unique id (*) in the site's .well-known/web-app-origin-association file. Other web apps cannot take advantage of this association, as they have different unique id.

Validation of associations should take place after the installed app is first launched. Validation should not succeed if the app is never online. If validation succeeds then afterwards the app remains offline, the associations will remain valid. Periodic revalidation takes place if the app is online. This is similar to how comparable schemes are implemented for Windows, Android, and iOS.

@torgo No dependency on IWA. I agree with @alancutter 's comment - i.e. there is no support here for fully offline apps that have no opportunity to do online validation with other sites.

(*) - https://github.com/philloooo/pwa-unique-id/blob/main/explainer.md

Comment by @cynthia Jan 25, 2024 (See Github)

Thanks for the response - who/what guarantees the uniqueness of the identifier? Without a single source of truth there isn't a way to guarantee uniqueness, and without that guarantee, it seems like we might have issues, such as enabling spoofing. Are we missing something?

Comment by @LuHuangMSFT Jan 26, 2024 (See Github)

Is the concern that https://siteb.com can spoof https://sitea.com by using the same app id? App IDs as described here and implemented in Chromium on desktop platforms are tied to app origin.

  • Web apps are served over https to be installable.
  • App ID is a function of the start_url origin, which must be the same origin as the document pointing to the web app manifest.

For an app from https://site-a.com to be installed with the same App ID as https://site-b.com, it would need to appear to the UA to have originated from https://site-b.com. Another possibility is that https://site-a.com changes ownership without the participating content origin's owner noticing.

I think it's possible for 2 separate apps from the same origin to claim the same app ID (start_url origin + specified ID) but we can understand this to mean there is only 1 app. Only one should be installable at a time.

Discussed Feb 1, 2024 (See Github)

Max: read this proposal... seems that we asked a question about uniqueness of the identifier -- they gave an answer abnout that...

Yves: it's not clear that IDs can't be spoofed - if it's a hash for example it's potentially fake-able. Also the scope extension is to extend the origin - the URL plus other origins... If the ID is tied to the URL then do they have multiple APP ids? I'll look at that more closely to see how they are doing that... If an app served from site-a,com wants to serve content rom site-b.com, site-b.com must be aware and accepting that... Maybe we could have them on a call?

Dan: I will reach out to Diego...

Discussed Feb 1, 2024 (See Github)

bumped to 11-march week

Discussed Mar 1, 2024 (See Github)

Yves: latest response difficult to understand.. their extension extends site A's capabilities to serve things from site B... That's where the problem is.

Matt: one of the questions raised was about uniqueness of IDs for apps - they've pointed to an explainer that talks about unique app ids... after that sangwhan was still concerned about guaranteeing the unique keys... they are claiming there can be a unique ID but there is some question about that.

<blockquote>

Hi @LuHuangMSFT - just coming back to this now. I think the risk is not necessarily that https://siteb.com can spoof https://sitea.com, but rather than since https://sitea.com can serve content from https://siteb.com as if it comes from https://sitea.com that content from https://siteb.com can therefore completely hijack https://sitea.com without the user's knowledge. So if that understanding is correct then that breaks the security guarantee for web content. The proposal would need to mitigate against this risk in some way way - for example, scoping this very tightly so that it cannot be abused in this way. We also think the spec needs a privacy review and we would suggest that you request that separately. Also, as Sangwhan mentioned, we remain concerned about the uniqueness of the identifier.

</blockquote>
Discussed Mar 1, 2024 (See Github)

Dan: no response to our feecback from last week.

Discussed Mar 1, 2024 (See Github)

Dan: pings Diego

Comment by @torgo Mar 13, 2024 (See Github)

Hi @LuHuangMSFT - just coming back to this now. I think the risk is not necessarily that https://siteb.com can spoof https://sitea.com, but rather than since https://sitea.com can serve content from https://siteb.com as if it comes from https://sitea.com that content from https://siteb.com can therefore completely hijack https://sitea.com without the user's knowledge. So if that understanding is correct then that breaks the security guarantee for web content. The proposal would need to mitigate against this risk in some way way - for example, scoping this very tightly so that it cannot be abused in this way. We also think the spec needs a privacy review and we would suggest that you request that separately. Also, as Sangwhan mentioned, we remain concerned about the uniqueness of the identifier.

Comment by @LuHuangMSFT Mar 28, 2024 (See Github)

since https://sitea.com/ can serve content from https://siteb.com/ as if it comes from https://sitea.com/ ...

I want to push back against saying that https://sitea.com/ would serve content from https://siteb.com. I would describe it as: both https://sitea.com/ and https://siteb.com/ can be displayed in an app window with the same app window treatment [1]. Either site would have its web contents displayed at the top level. Neither is enclosed in a frame or webview.

[1] In our implementation in Chromium, both https://sitea.com/ and https://siteb.com/ have their origin briefly displayed in the title bar. Also, origin information is clearly displayed in the window options menu. Do these UI treatments sufficiently mitigate the issue that users may be unaware of where the currently viewed content comes from?

There is a question of whether to users, the installed web app (which includes the recognizable app window, taskbar pins, shortcuts, and other assets) is equivalent to the site from which it was installed. If the sitea app is distinct from https://sitea.com, it wouldn't necessarily seem strange to also sometimes display content from https://siteb.com (provided users are sufficiently informed of the origin transition during navigations.)

In the same example, https://sitea.com explicitly says in its web app manifest that when installed, it allows the app window to host content from https://siteb.com. This is also an explicit endorsement of the content of https://siteb.com/ to the user.

What does "hijack" mean in this context? How is the security guarantee for web content broken?

Discussed Apr 1, 2024 (See Github)

Dan: we analyze LuHuang's comment

Matthew: a couple of things from that and also from Daniel Murphy - "there is a 2 way handshake that is there to ensure that both origins are cooperating." ... also "This does NOT show content from one origin on another origin" They give a concrete example with zoom...

we read through Daniel's zoom example

Yves: does "in scope" mean sharing things like array buffers or even cookies?

Yves: the proposal is not only about subdomains - but about any other domain...

<blockquote> Hi folks - I think we can best progress this review with a call where we can talk through the proposed use cases a bit more interactively. Given that we have the W3C Advisory Committee meeting coming up, I'd like to suggest that we hold this call in our of our TAG breakouts the week of the 22nd of April. I'll follow up by email to arrange. </blockquote>
Discussed Apr 1, 2024 (See Github)

Lu Huang: web app windows do this thing when you navigate outside the scope - or outside the origin - a big warning bar - the warning "you've gone somewhere else" the button "do you want to go back"? Some people say this is not desirable... because the webapp owner also owns the other domain... You can tell your webapp that these origins are also aprt of the app.

Dan Murphy: concrete examples - if you've tried to use slack as an installable webapp - there's always an issue with different origins for multiple orgs...

Tess: i don't think that happens when you add slack to a webapp on the doc in safari in macos....

Dan Murphu: slack may have changed... maybe safar is operating in a different way .. i know safari doesn't show the banner for login pages... when to do out of safari and when to stay in the app... the way that scope is defined needs to be in the same origin no way for it to be visible without

Dan: Embedded content as well?

Lu Huang: to me, it's similar to being not in a webapp window just in a browswer window... you navigate to another site and no alarm bells go off. For the implementation in chromium we show the user the (new) orgiin and also have security & origin info in the main menu in the webapp window... In this issue what

Lu Huang: iframes would work exactly the same way - everything we're talking about here is top level navigation... even then when you navitage to the new site it doesn't inheret any preferences...

Matthew: slightly related: we've been having conversations about heuristics ... might be something in relation to that...

Dan Murphy: there's product and then web platform... if there's a navigate=self link that navigates to the current document... this feature is all about the primary browsing context... the web content is in this other frame... Worst case scenario might be: a link in mastodon... there's a link in that page to another origin and it's a navigate=self link... user clicks on that, it goes to a different origin,... worst case scenario is .. mastodon.com origin says x.com is part of me, x.com says mastocon.com is part of me... In that case no notification would show...

Yves: what is the implication of considering that the application encompass other scope? sharing data between different.

Lu Huang: no tha's not part of the proposal...

Peter: is that always going to be clear to the user? I start out on one site and I move to another site that is related? Is that going to be clear to the user? Even if that's not the intent then we have a concern that someone will try to

Dan Murphy: the requirement for this feature is establishing an edge on a graph of origins - in a way that is safe... the feature needs that so we don't show the security boundary...

Peter: if you navigate to another origin and it's not displaying a URL bar...

Lu Huang: usually a clear origin that is asking for microphone or location permission... The UI should always make it clear. That's a reasonable thing to recommend in spec language. Also a way for the user to look up where am I...

Dan: Murphy - we can say "this is not an extentison of storage, or permission" ..

Peter: still concerned - the permissions case - most users don't understand the concept of origin so we need to be careful how we expose... maybe we should double-key these permissions...

Lea: reading the goals it reminded me of first party sets, related website sets...

Lu Huang: it's been suggested we look at that but related website sets does too much - has security implications we don't want. It works at the domain level. We can't use it to put together a set of subdomains.. For this set we have tried to not change assumptions aboue security, permissions, storage... not changing the model at all. Strictly talking about webapp features... The one effect we're talking about ... just the UI around the cct - custom tab bar - warns you that you're out of scope... Maybe there reasonable other webapp features to add to this...

Lea: my concern is that we don't want a bunch of differen to APIs to declare the same intent... should only declare this once... should not have to scatter this through multiple different APIs...

Dan Murphy: this is constructed for "manifest entities" a manifest isn't a whole origin it's often a subset of an origin... Really it's just a message to the user agent... file handlers is an example of a feautre added to manifest... I agree it's annoying that there are too many concepts. This one is specific to manifest files... tied to a specific app... ID that is specified there... In terms of furture APIs ... don't see any APIs that it makes sense for.. Chrome may have wanted to ehhance the user experienced based on this manifest entity...

Lea: once this ships if it enables certain permissions... scope has a certain meaning... Concerned about the increase in API sufrace area for declaring similar things.

Yves: if you have different origins

Lea: looks like this also specifies domains but also specifies subdomains as well?

Lu Huang: right now the entries are origins... we want it to be able to support more specific path filtering... we don't know what the right syntax is... URLpattern is supposed to solve that. but does it translate well to native for differnt to operating systems? Does it translate to OS specific registrations. We split it into two problems..

Lea: right now all the examples related to URLs... is there an example that shows an OS specific scope?

Lu Huang: as it stands web app scope is single origin and some path with the single origin.. if we make webapp scope more complicated then it's whether you can translate it to things you care about in operating systems... future concerns... that's why we didn't include a filtering syntax...

Lea: but wildcards are such a syntax.. if anything it seems to me that example under number 1 that could be a single URL pattern...

Yves: in service worker the patterns are similar for performance reasons...

Lea: we don't want multiple related patterns... different microsyntaxes for URL patterns...

Lu Huang: good point..

Dan Murphy: no way to parse that syntax right now .. our feedback was to make a separate entry for eTLD+1... happy to have other ways... it should be parsable by spec... simplest way to do what our partners need...

Lea: if the pattern is that URLpattern is more broad... you could just say that's it's a URLpattern with only host or only origin keyword... you can define that pattern as a hostname pattern...

Lu Huang: the examples are meant to be origins... the scheme is assumed to be https given that webapps only care about https urls... even if we set the same limitations - should only be an origin - we can still express that in terms of URLpattern...

Lea: yes because users can learn that only once... my cocnerns are (a) the patterns and (b) yet another API to

Dan: developer complexity

Lea: for smaller teams if you have to maintain a list of domains in different places... concerned about keeping those in scope...

Dan: other than fps ... related web sites .. is there anything else?

Lea: maybe a a CORS extentension? actual intent authors are trying to express ... how does this web site relate to other origins.. can they navigate, install, etc...

Lu Huang: CORS extension has not come up... I wonder how it would make a relationship between an APP ID and an origin... scope extensions can be split into 2 parts... what you do with it and what the webapp manifest says... not married to a brand new association format, but of the things we looked at there wasn't one that satisfied our requirements... we can have this without being fixed on one things...

Lea: this seems to keep coming up so maybe work with the first party sets people....

Dan: webapp manifest files vs. http headers

Lu Huang: I think the maifest is the right surface to have that... I think our discussion ... for those origins to confirm they want to take part in this... do they want a .well-known config file... cors header... the related website thing doesn't work... heavy burden... you have to apply to get your configurations...

Lea: i think the permissions should be explicit... Right now as someone reading the manifest ... unclear what these permissions are... can they change... what if the API made this explicit... these origins share these types of permissions...

Lu Huang: tagging syntax... can we also add tags there... the syntax is extensible...

Dan Murphy: I think we want to be ware of permission grant... my interpretation of the feature is that this tells UAs about a feature.. but could also be useful for install...

Discussed Apr 1, 2024 (See Github)

Dan: just noting the feedback from Lu Huang - they are going to do an update on the explainer... lookling at cors, looking at URL patterns...

Matthew: interesting difference in opinion of the spec between this group and what Apple /webkit has done.

Dan: the use case itself is clearly cross platform...

Dan: I think at this point we're waiting for an update to the explainer...

Matthew: one other thing I noticed - i don't think they have an "alternatives considered". so they maybe should add one.

<blockquote> Thanks @LuHuangMSFT - let us know when you have updated the explainer and we'll re-review. Much appreciated. I'd like to encourage you to include more info on abuse cases - and mitigations against these. Can you also tighten up the scope to the problem you're trying to solve and explicitly exclude things like permissions, local storage sharing, etc... Can you also please add an "alternatives considered" section of the explainer with some of the alternatives that we discussed in the call? </blockquote>

sent

Comment by @dmurph Apr 2, 2024 (See Github)

Voicing support for scope_extensions here from Google. This is a very important use-case for web developers trying to enable app experiences on the web platform, and comes from how organizations often don't use just a single origin for their product.

Feedback from @torgo: I believe this is already scoped very tightly:

  • This only affects logic that asks if a given url is "in-scope" of an app. This is done currently in one place - when we choose whether or not to show the toolbar in web apps showing an 'out of scope' origin. Please see how this works with this site - if you install this site in Chrome or Edge, clicking on either of those links will demonstrate this toolbar. With scope extensions, the 'secondary' link will no longer show that toolbar.
  • There is a two-way handshake here, demonstrating the ownership of both origins here by the app entity.
  • This does NOT show content from one origin on another origin. When opening the url here back in Chrome or Edge, all is correct.

Illustrative example:

  • Zoom is at www.zoom.com
  • Zoom has an app installed there, the manifest entity is rooted to that origin (start_url / manifest_id are at that origin).
  • Zoom also owns all other *.zoom.com subdomains, and each organization gets a subdomain.
  • Currently, all zoom links for customers, when opened in the app, show the toolbar.
  • This feature allows them to assert they own these subdomains and the manifest entity can consider those "in-scope", and not show that toolbar for their owned subdomains or other origins they own.

Feedback for @cynthia:

  • The identifier here is spec'd and comes from the manifest entity of the app.
  • This uniquely identifies the app, and the 'origin' part of the identifier MUST match the origin of the document that links to this manifest.
  • This means that it is impossible for an app to replicate an identifier for an origin they do not own. For an identifier to be 'https://www.a.com/id', this MUST have been from a document on that origin that has a manifest link (like <link rel="manifest" href="manifest.json" />) to a manifest to create that manifest entity. We can talk more about parsing options here, but the main guarantee here is that origin must match the document origin.
  • Thus - it is safe to assume that specifying an identifier here with a given origin means that the app referenced there must be tightly coupled to that origin, and was defined by & linked from a document on that origin.

Hopefully those constraints are enough to make this reasonable?

More abstractly - the feature is communicating & verifying 'what other origins are part of this app', so the user agent can work for the user accordingly.

Comment by @torgo Apr 3, 2024 (See Github)

Hi folks - I think we can best progress this review with a call where we can talk through the proposed use cases a bit more interactively. Given that we have the W3C Advisory Committee meeting coming up, I'd like to suggest that we hold this call in our of our TAG breakouts the week of the 22nd of April. I'll follow up by email to arrange.

Comment by @hober Apr 16, 2024 (See Github)
  • This only affects logic that asks if a given url is "in-scope" of an app. This is done currently in one place - when we choose whether or not to show the toolbar in web apps showing an 'out of scope' origin. Please see how this works with this site - if you install this site in Chrome or Edge, clicking on either of those links will demonstrate this toolbar. With scope extensions, the 'secondary' link will no longer show that toolbar.

How relevant is this feature outside of the Chromium project? When I add the example site to the Dock in Safari on macOS, I don't see whatever toolbar you're talking about.

Comment by @LuHuangMSFT Apr 16, 2024 (See Github)

Thank you to all who gave feedback at the breakout session.

I'm pleased we were able to address concerns about storage/cookies/permissions/preferences remaining separate by origin. We can and will call that out more clearly in the explainer.

We will also look into whether the information conveyed through the existing manifest field + web app origin association file can be expressed using feature-restricted URLPattern.

We will do some research on CORS extensions and reply with pros and cons of using this instead of web app origin association. Associated Website Sets is unsuitable as it operates on full domains instead of origins and does not have the primitives to select between different web app IDs on the same domain for association.

I do support the high-level goal of not introducing multiple ways of configuring similar things to the web platforms. I'm ready to consider and do due diligence on any alternative association mechanisms that fulfil our requirements for web apps. I want to reiterate that it seems reasonable to me for a web first app-to-website association format to exist (similar to windows-web-app-link, assetlinks.json, etc.) as app-to-website association has some different requirements from website-to-website association.

Comment by @alancutter Apr 17, 2024 (See Github)

How relevant is this feature outside of the Chromium project? When I add the example site to the Dock in Safari on macOS, I don't see whatever toolbar you're talking about.

UI when navigating out of scope is mentioned in the manifest spec: https://www.w3.org/TR/appmanifest/#nav-scope

If the application context's active document's URL is not within scope of the application context's processed manifest, the user agent SHOULD show a prominent UI element indicating the URL or at least its origin, including whether it is served over a secure connection. This UI SHOULD differ from any UI used when the URL is within scope of the application context's processed manifest, in order to make it obvious that the user is navigating off scope.

Comment by @tomayac Apr 17, 2024 (See Github)

How relevant is this feature outside of the Chromium project? When I add the example site to the Dock in Safari on macOS, I don't see whatever toolbar you're talking about.

macOS Safari just opens out-of-scope links in the default browser (nice touch!). It's relevant on iOS and iPadOS, where you do see an in-app browser with the Done button in the upper left corner.

Comment by @torgo Apr 24, 2024 (See Github)

Thanks @LuHuangMSFT - let us know when you have updated the explainer and we'll re-review. Much appreciated. I'd like to encourage you to include more info on abuse cases - and mitigations against these. Can you also tighten up the scope to the problem you're trying to solve and explicitly exclude things like permissions, local storage sharing, etc... Can you also please add an "alternatives considered" section of the explainer with some of the alternatives that we discussed in the call?

Discussed Jun 1, 2024 (See Github)
Discussed Jun 1, 2024 (See Github)

We're waiting for an update on the explainer, no activity yet.

Discussed Jun 1, 2024 (See Github)

Dan: They updated their explainer - considering that they have been responsive to the review comments I'm pretty positive...

Martin: do they need the ability to take an entire registerable domain?

Yves: there is a scope in it that is the path ....

Martin: it looks like a scope and scope extensions ... "the following origins are also included" - does that mean just /app under those domains or anything else in that domain... that's unclear to me. Does it meaan that if you want to extend your scope to another origin then you have to follow the same path construction rules?

Yves: weird if you want to import some images ...

Martin: i think this is for the top level navigations... you'd stay with the idea that you are in a particular app in the case that you navigate to another origin... Any subresouces on a top level page would be fine... This is all about navigation.. if there's a link in the page that takes you to another origin do you take them "to the browser" or within in the app.

Dan: i think the main point is to not show a warning ... if navigating to another origin within an app

Yves: linked to caching?

Martin: it shouldn't be - it's "im in an app - and i navigate to another site... "

Martin: another question - the external website doesn't have much of a say of how much the "app" is able to use of it... say you have a payment flow that uses a payment provider... the payment provider stays in app.. but if you've nominated the entire origin - say the payment's provider's home page - might stay in the app which could break the user experience...

Dan: e.g. you're in a payment flow and then you want to view the privacy policy of that payment provider...

Martin: the external web site might want to be able to say "this app can use the following URLs ... otherwise jump out and it's no longer in the app" there's a discussion in the explainer about authorizing with an association file - it only associates with an origin - and doesn't recognize the idea of multiple app...

Yves: in the explainer it's in "future extensions"... so maybe the comment should be that this shouldn't be a future example

Martin: We have a scope parameter on the manifest in the first place... It might be a set of prefixes.

Yves: needs to be in sync with what is done with service worker...

Martin: path prefix thing with a pattern...

Comment by @diekus Jun 11, 2024 (See Github)

Hola TAG,

We've updated the explainer to provide a better sense of the feature we are developing:

  • added images that show the security UX that scope_extensions addresses
  • added wording that explicitly states there are NO changes to permissions on extended scopes (every origin is in control of their own permissions)
  • included an example of how an installed web app can always show the user the current scope where content is being served from (for security)
  • changed the way a developer can specify these extended scopes by re-using an existing concept (that of registerable domains) taking into consideration the feedback we got in our breakout review.

We hope this addresses the concerns from the previous review and thank you for all the help getting to this current revision.

cc @dmurph @LuHuangMSFT

Comment by @martinthomson Jun 19, 2024 (See Github)

Those people who have been involved asked me to thank you for being responsive to feedback. This is very much appreciated.

Some questions from just me, from looking at this with somewhat fresh eyes:

Scopes are a path prefix on the current origin. This adds two types of scope extension: origin and registrable domain, but it is not clear to me what the concrete test is. Presumably, when a navigation link is presented in the "app", the container needs to make a call about whether this is an "in-app" navigation or it is "leaving the app" (which might result in a warning or opening the URL in the "main browser"). Currently, the test is url.startsWith(origin + scope). This extends that to include some number of additional tests, but these are not immediately obvious.

origin: is this the ENTIRE origin? If you imagine an external service that provides a limited service, it might be preferable to specify a limited scope on that origin. (Obviously, you can't just use the scope specified in the manifest, because it won't make sense on a different origin.)

registrable domain: I think that you want "site" here and want to use a same-site test for testing. This creates a broader match set that needs to be authorized on a per-origin basis, which is fine, but it has the same problem regarding scope on a per-origin basis. (Naming this "site" might be a better naming choice, with the value being a host rather than what appears to be an origin in the explainer. The scheme can be implicitly HTTPS.)

Your future extensions mentions:

More specific scoping e.g. scope suffix or include/exclude lists or URL patterns.

We tend to think that this is necessary - in two directions. That is, both the app and the new origin should be able to specify which resources are included, with the end result being the intersection of those sets (each could leave it unspecified if that is their choice).

@ylafon suggests URL patterns for this rather than path prefixes, though I'm personally less firm about that, because prefixes are easier to understand. The manifest has a path prefix in scope, which makes sense there, mostly because we didn't have URL patterns at the time. A URL pattern is pretty reasonable if you think it is adequately comprehensible.

Also, we observe that a single origin can include multiple apps with different manifests, but the authorization from the new site does not -- and cannot -- identify a single app. It's reasonable to say "I authorize any app", but it might be better to have it the authorization be scoped to specific apps, which might have different authorized scopes.

That might lead to an authorization like this, for a hypothetical payment provider:

{
  "web_apps": [
    {
      "site": "foo.example",
      "scope": "/payments/*",
    },
    {
      "origin": "https://bar.example.net",
      "scope": "/payments/*",
    },
    {
      "manifest": "https://example.com/app/manifest.json",
      "scope": "/payments/*",
    }
  ]
}

Another nit, but the whole { type: "foo", value: "bar" } construction is a bit redundant, how about making this simpler?

{
  "scope_extensions": [
    { "site": "example.com", "scope": "/*" },
    { "site": "foo.example", "scope": "/that_app/*" },
    { "origin": "https://bar.example.net", "scope": "/that_app/*" },
  ]
}
Comment by @LuHuangMSFT Jun 28, 2024 (See Github)

Thank you for the fresh review.

Current test: url.startsWith(origin + scope)

New test: url.startsWith(origin + scope) OR url.startsWith(any origin in scope_extensions) OR registrableDomain(url) == registrable domain in scope_extensions

origin: This is the entire origin where there is no additional scoping filter.

registrable domain: replacing this with "site" and using the same-site test seems like a viable option. We still want to allow the developer to be able to provide a single origin association file at the manifest-provided site to validate the scope extension. @dmurph what do you think?

Scope filtering

Allowing one or both sides (app manifest and origin/site) to filter the scope using one or more kind of filtering syntax is a good idea. "...each could leave it unspecified if that is their choice" is what we want to start with. This is restrictive for app devs initially but we believe it will unblock many developer scenarios and allow us to better understand developer needs and design requirements for filtering.

I think scope suffix in the origin association file is the most suitable first filtering syntax to implement as developers are already familiar with how this works and it is a fast test to perform. What do you think about specifying this even if it is not implemented initially or would be it better to leave the specification to future work?

There is a shared problem with a.) filtering from both sides and calculating the intersection and b.) using URLPattern: performance when matching URLs and the ability to convert the filter to OS-specific filtering syntax. The latter is necessary for OS integration features like URL handling where the OS performs the URL filtering. Both a.) and b.) could lead to slower filtering due to their complexity. Both a.) and b.) could be difficult to convert to OS-specific filtering syntax.

I still think specifying no filtering syntax or just suffix scope match initially is the best way forward until we do the work to study the performance and OS compatibility of filtering syntax options (URLPattern, etc).

Validation from origin/site configuration

The .well-known/web-app-origin-association file hosted by the participating origin identifies individual apps by their app id (web_app_identity in the explainer example):

{
  "web_apps": [{
    "web_app_identity": "https://example.com/"
  }, {
    "web_app_identity": "https://associated.site.com/"
  }]
}

Each object can be extended to apply different scope filtering for each app independently.

Type: Value

This isn't strictly necessary but makes parsing slightly more convenient as type is a required string field and the whole object can be skipped if type is not found or it is not a recognized type in this UA.

Comment by @dmurph Jun 28, 2024 (See Github)

Hello!

A few thoughts:

@martinthomson said:

origin: is this the ENTIRE origin? If you imagine an external service that provides a limited service, it might be preferable to specify a limited scope on that origin. (Obviously, you can't just use the scope specified in the manifest, because it won't make sense on a different origin.)

(later)

Another nit, but the whole { type: "foo", value: "bar" } construction is a bit redundant, how about making this simpler?

AFAIK this has been the only request from our partners. I think it would be reasonable to add a 'scope' parameter to the 'origin' type, or a new type. I worry about using a "bag of parameters" here as it can lead to complications if more options are added & trying to figure out how they might combine. (AKA - what if you specify both origin & site? we would have to specify what this behavior means, and future options increase the cross-product complexity). It's clearer to have this format:

"scope extensions": {
    { "type": "site", "value": "https://example.co.uk" },
    { "type": "origin", "value": "https://helpcenter.example-help-center.com" }
    { "type": "url_prefix", "value": "https://my_github_project.github.io/production/" }
}

(or - add a 'path_prefix' member for 'origin' types). Feel free to bikeshed "url_prefix". But this has not been a request from developers. If you feel strongly about needing this, that type is not complex to implement. Lu also mentioned better for cross-user-agent capabilities if this needs more - the functionality is based on the type, not based on arbitrary options.

@martinthomson said:

registrable domain: I think that you want "site" here and want to use a same-site test for testing. This creates a broader match set that needs to be authorized on a per-origin basis, which is fine, but it has the same problem regarding scope on a per-origin basis. (Naming this "site" might be a better naming choice, with the value being a host rather than what appears to be an origin in the explainer. The scheme can be implicitly HTTPS.)

This sounds good to me - the important part is that we use the public suffixes list, which that algorithm uses through obtaining the site.

@martinthomson

We tend to think that this is necessary - in two directions. That is, both the app and the new origin should be able to specify which resources are included, with the end result being the intersection of those sets (each could leave it unspecified if that is their choice).

As far as I know, this has definitely not been a request from partners. I worry this isn't necessary and increases the complexity of this feature. It also seems not necessary - the web-app-origin-association file is used to convey ownership / two way handshake of the site. With this established, the manifest is trusted to specify what it needs.

@martinthomson said:

Also, we observe that a single origin can include multiple apps with different manifests, but the authorization from the new site does not -- and cannot -- identify a single app. It's reasonable to say "I authorize any app", but it might be better to have it the authorization be scoped to specific apps, which might have different authorized scopes.

As Lu pointed out above, this is not the case. The web app identities are listed explicitly in the web-app-origin-association file.

Discussed Jul 1, 2024 (See Github)

Yves: planned as a future extention but we are discussing doing it now...

Dan: looks like a productive discussion... Martin needs to be involved in moving this forward...

Comment by @martinthomson Jul 1, 2024 (See Github)

I worry about using a "bag of parameters" here as it can lead to complications if more options are added & trying to figure out how they might combine.

That's easy. They all apply. If you have { origin: "https://example.com", site: "example.co.uk" }, then it doesn't match. But it leaves you the option of saying { origin: "https://example.com", scope: "/payments" } or similar.

this has definitely not been a request from partners

Your partners are not the only stakeholders here. Or maybe they didn't think that this was a problem for them.

The concern here is that a manifest on a different site could cause content from a completely different site to be included in that app. If a service accepts being part of an app, it loses granular control over which resources are included in that way. Think about X-Frame-Options or CSP's frame-ancestors rules. These can prevent content from being included as part of someone else's site in frames. Those can be specified on a per-resource basis, so that you can protect specific resources as necessary. Here, the choice is origin-wide, removing that option. I have no doubt that providing a unique origin for each partner is an option that some services will choose to exercise, but forcing that choice seems unnecessary when the fix is so trivial.

Comment by @LuHuangMSFT Jul 2, 2024 (See Github)

I'm flexible on:

  • Using type vs. bag of parameters. type seems easier for developers to interpret, but bag of parameters could also be specified strictly to the same effect (where developers don't have to guess which params are compatible.)
  • Starting with a scope suffix per site/origin object (in the web-app-origin-association file) if this is sufficient to address concerns. Applying a scope to multiple origins that pass the same-site test places restrictions on the developer as the scope needs to make sense for all origins that pass the same-site test with their desired site.
Comment by @martinthomson Jul 3, 2024 (See Github)

Starting with a scope suffix per site/origin object (in the web-app-origin-association file) if this is sufficient to address concerns. Applying a scope to multiple origins that pass the same-site test places restrictions on the developer as the scope needs to make sense for all origins that pass the same-site test with their desired site.

I wasn't suggesting that the app manifest specify a scope. That is, you would still just say {"site": "service.provider.example"} in the manifest of the app that is trying to extend the app scope to the service provider origin. As you say, if you said {"site": "service.provider.example", "scope": "/foo"}, you are forcing multiple origins to all have the same structure. Then, though "app1.service.provider.example" might have "https://app1.service.provider.example/foo", "app2.service.provider.example" either has to accept the same conditions on the use of that scope or not use "/foo", which is bad either way.

But that is the app provider imposing its will on the service providers, which is basically an RFC 8820 violation. You want the service provider to speak for itself.

Instead, the opt-in from the service provider would list the apps that are authorized for use, plus a scope. That is naturally origin-scoped anyway. {"web_apps": [{"web_app_identity": "https://example.com/", "scope": "/foo"}]}, coming from "https://app1.service.provider.example/" would have the desired effect. And then "app1.service.provider.example" can make its own choice about what to include (or not), which will be nothing by default.

Comment by @LuHuangMSFT Jul 3, 2024 (See Github)

Instead, the opt-in from the service provider would list the apps that are authorized for use, plus a scope. That is naturally origin-scoped anyway. {"web_apps": [{"web_app_identity": "https://example.com/", "scope": "/foo"}]}, coming from "https://app1.service.provider.example/" would have the desired effect. And then "app1.service.provider.example" can make its own choice about what to include (or not), which will be nothing by default.

I think we're in agreement. The .well-known/web-app-origin-association file hosted by the origin/site should look like:

{"web_apps": [{"web_app_identity": "https://example.com/", "scope": "/foo"}]}

with scope being optional.

Above, I wrote:

We still want to allow the developer to be able to provide a single origin association file at the manifest-provided site to validate the scope extension.

I was trying to point out that if the developer uses a single web-app-origin-association file for multiple origins that pass a same-site test means that if it also specifies scope then scope applies to all origins that pass the same-site test.

Comment by @martinthomson Jul 4, 2024 (See Github)

I was trying to point out that if the developer uses a single web-app-origin-association file for multiple origins that pass a same-site test means that if it also specifies scope then scope applies to all origins that pass the same-site test.

Just because this is a little murky still. My point is that only an origin can speak for itself in this regard. That is, "https://a.example.com/" (an origin) can't speak for "https://b.example.com/" (a different origin), even if they are the same site.

Of course, if you have operational practices that mean you put the same file at the well-known location on both of those origins, that's fine, but that's not our business.

Comment by @LuHuangMSFT Jul 5, 2024 (See Github)

In the scenario I'm trying to address, the file is at https://example.com/.well-known/web-app-origin-association and it would get applied for both https://a.example.com and https://b.example.com.

Comment by @martinthomson Jul 6, 2024 (See Github)

OK, I don't think that is a good goal. Every origin has a different set of resources. Having one origin able to speak for others, even if it is same-site with those origins, breaks the fundamental scoping properties that underpin origins.

Yes, we have a privacy boundary at the site level and a bunch of stuff sort of expands to fill that scope by virtue of having some interaction with cookies. This does not.

Comment by @dmurph Jul 8, 2024 (See Github)

I agree that it's not great, but this is unfortunately a requirement due to the existing way the web is set up. The Zoom use-case is the best example here, but other companies are set up similarly:

  • The Zoom web app is at https://app.zoom.us/
  • Each organization that contracts Zoom has a custom sub-domain, like "https://mycompany.zoom.com"
  • Zoom wants to just have one app that can be used for any organization, where the use would use the same app to join zoom calls with any organization they are part of or given a meeting link to.
  • Without the same-site capability, zoom would need to list every single company they contract with in their manifest in their manifest, which is both a infeasible size issue and a privacy / data leak issue for them (and host a web-app-origin-association file in each of those domains).

So unfortunately allowing an organization / entity to say "any urls in this site can be considered part of my app" is a required use-case here that we can't remove :(

This is one of the reasons feature is very tightly scoped to just the 'scope' evaluation of a web app (e.g. what pages can be considered as part of this app, owned by the same company / entity). This matches what 'same-sites' mean - including the registrable domains check. In the last TAG meeting this was actually brought up as a potential issue - being too tightly scoped. However due to this same-site check this is one of the reasons it was kept as such a tight scope - no other feature should be able to use this.

So then there are two questions at the end of this:

  • Is this same-site check something that seems ok as it's so narrowly used for just this specific feature?
  • Does the rest of the API seem OK, for the other use cases that don't need same-site and only need origin checks?
Comment by @LuHuangMSFT Jul 9, 2024 (See Github)

Another reason not to require each origin that passes the same-site check to host a copy of the validation file is that the UA can only validate this information at navigation time and not when the web app is installed.

At the time the web app is installed, the UA needs to be able to validate the web-app-origin-association file from a finite list of origins in order to support behaviors such as:

  • URL handling where configuration needs to be written to the OS.
  • Deciding whether to display warning UI promptly instead of "popping" it into the visual layout after navigation is complete.
Comment by @reillyeon Jul 23, 2024 (See Github)

@martinthomson, I'm just a bystander on this proposal but reading through your comments it sounds like you are generally comfortable with the idea of an origin opting-in to a scope extension from a cross-origin app but are specifically concerned with the ability for a site (registrable domain) to do so on behalf of the origins under it. This makes sense to me.

I am curious what you think of using a new pair of HTTP headers (e.g. App-Id and Allow-App-Scope-Extension) in a similar way to the Origin and Allow-Cross-Origin-Access we have for CORS. This would enable browsers to "trust but verify" on a site-level scope extension by checking that the server for the particular origin being navigated to agrees it wants a resource to be considered part of the app. This has the advantage over requiring each origin to host a copy of the .well-known/web-app-origin-association file that it provides the information in-line with the request that is already happening and is thus easier for browsers to implement and doesn't introduce additional navigation latency.

There are some details to work out but I think this would maintain the ability for the browser to do the necessary install-time setup for scope extensions while keeping ultimate control over how their resources are presented in the hands of the origin.

Discussed Aug 1, 2024 (See Github)

Martin: feedback was let the web site in the scope have something to say about it... they wanted to do this on an origin-wide basis... Idea is that you have a webapp that includes stuff from outside the origin of that webapp - so the webapp declares please include this other stuff in my scope. The request from the feedback we gave is to give the other origin fine-grained control of what resources are in the app vs. not in the app... Reilly suggests a "CORS" mechanism - and trust but verify... Potentially too late at the point you've started to make requests though...

Martin: opt-in suggested in original explainer - at the target origin you would host a resource that says "these apps are ok on this site for all origins"...

Dan: a new type of resource that would be at a well-known location...

Martin: It's a reasonable thing to be able to do... Reilly is suggesting that the manifest sets the scope and that when you navigate, that navigation request uses a cors-like protocol that says "the following app has asked me to navigate to this resource" then the resource itself says "ok" (in a header). Don't know if it's reasonable to do this in the navigation and loading algs...

Martin: a file can be relatively straightforward...

Dan: https://example.co.uk/.well-known/web-app-origin-association (from explainer)

Dan: can we leave feedback to the tune of

<blockquote>

Hi @LuHuangMSFT - as noted we are fine with the user need and we are generally fine with the design. @reillyeon has suggested a CORS or CORS-like approach. Whilst that has some advantages (in particular, not requiring a new well-known located file) it also has the disadvantage of requiring complex server configuration. Our main concern stands, however, that the Association file should allow for more specifying resources in a more fine-grained way. This could include wildcards to make it easy to create a very permissive Association file, but it should be possible to lock down the resources that are OK to share. Would you be OK with modifying the design to allow for such an approach? If so, we're happy to close this review as satisfied.

</blockquote>

Matthew: urlpattern as also brought up

Martin: I think it's a secondary issue...

some discussion on install cross-origin

Dan: Leaves comment

Discussed Aug 1, 2024 (See Github)

Dan: I reached out to Diego to ask him to help clarify whether 👍 means "yes" to our proposal. If so, I suggest we close this as satisfied at the plenary.

Comment by @martinthomson Aug 7, 2024 (See Github)

Interesting idea. How would that be realized though? You would send the target resource the App-Id, have it recognize that it is being framed it, then accept that with a field (Allow-App-Scope-Extension).

That would work in that it is sufficiently granular, but I have two concerns:

  1. Operationally, that seems more difficult to manage than the approach that @LuHuangMSFT and @dmurph are advocating for.

  2. As a practical matter, at the time that the request is made, a lot of stuff about a request has been predetermined (things like Sec-Fetch-Dest). I'm not sure whether a request for navigation in an app (i.e., what would happen if the resource is in the app scope) would differ from navigation in a browser (i.e., what would happen if it were not). Having to pick one and then back out if it fails would be good to avoid.

Comment by @torgo Aug 7, 2024 (See Github)

Hi @LuHuangMSFT - following on from what @martinthomson said above: as noted we are fine with the user need and we are generally fine with the design. @reillyeon has suggested a CORS or CORS-like approach. Whilst that has some advantages (in particular, not requiring a new well-known located file) it also has the disadvantage of requiring complex server configuration. Our main concern stands, however, that the Association file should allow for more specifying resources in a more fine-grained way. This could include wildcards to make it easy to create a very permissive Association file, but it should be possible to lock down the resources that are OK to share. Would you be OK with modifying the design to allow for such an approach? If so, we're happy to close this review as satisfied.

Comment by @LuHuangMSFT Aug 14, 2024 (See Github)

Update: I am investigating making 2 modifications to the design.

  1. For the case where the same-site format is used in the web app manifest to include a dynamic number of origins, add an additional requirement that response headers from an origin must include the manifest IDs of apps it agrees to be part of.

  2. Allow sites/origins to provide a 'scope' filter in their association files in .well-known path. This allows controlling resources using the same syntax as what is used in manifest scope.

I will update with more details after my investigation is complete.

Comment by @LuHuangMSFT Aug 20, 2024 (See Github)

Specifying scope in the association file

I've looked at the following options to give the association file a way to specify resources explicitly. My preference is for the fallback list. It's more verbose but allows us to accept URLPattern (or other formats) in the future. UAs are able to fall back to a simpler format following a clear order if they are unable to process a more complicated format. If left unspecified, the whole origin/site is considered part of the identified app with no resource restrictions.

Bag of things 1

{
  "web_apps": [
    {
      "web_app_identity": "https://myapp.com/index.html",
      "scope": "/app",  // Can be string or array of strings.
      "scope_url_pattern": "https://this-origin.com/?"  // Can be string or object.
      // Extend here.
    }
  ]
}

Bag of things 2

{
  "web_apps": [
    {
      "web_app_identity": "https://myapp.com/index.html",
      "scope": {
        "prefix": "/app",  // Can be string or array of strings.
        "url_pattern": "https://this-origin.com/?" // Can be string or object.
        // Extend here.
      }
    }
  ]
}

Fallback list

{
  "web_apps": [
    {
      "web_app_identity": "https://myapp.com/index.html",
      "scope": [
        {
          "url_pattern": "https://this-origin.com/?"  // Can be string or object.
        },
        {
          "prefix": "/app",  // Can be string or array of strings.
        }
        // Extend here.
      ]
    }
  ]
}
Comment by @LuHuangMSFT Aug 20, 2024 (See Github)

Request and Response Header

For multiple sub-domains included from the web app manifest using a same-site entry, we can make use of a request and response header design with no preflight. This allows an origin that was included via the same-site manifest entry to confirm its participation. This doesn't require an association file to be fetched immediately before fetching the resource and would not slow down navigation. This is largely what @reillyeon described above.

Example:

App window navigates to https://foo.com

  • Request contains App-Id: https://myapp.com/index.html header.
  • Response contains App-Scope-Extension-Allow-Id: https://myapp.com/index.html, https://otherapp.com/index.html

The resource can load in the app window without warning UI or being moved to a tab. The response can be App-Scope-Extension-Allow-Id: * to match all app ids. If the app id does not match or there is no matching response, the resource is not treated as part of the app.

The server can either configure a static list of app ids for simplicity or dynamically control the value of App-Scope-Extension-Allow-Id and use this to implement scoping.

Comment by @torgo Aug 29, 2024 (See Github)

Hi @LuHuangMSFT thanks for this - we're just reviewing in our TAG plenary call today and we're going to get back to you after we have a chance to think about / discuss more deeply.

Comment by @dmurph Aug 29, 2024 (See Github)

Question - do we need to have the initial App-Id header? Can we only have the second one? The implementation of that seems possibly infeasible.

Comment by @martinthomson Aug 29, 2024 (See Github)

So I think that the requirement needs to be that the browser needs to know whether something is in app scope before navigating. Any solution that involves finding out at the time is going to introduce latency penalties that are undesirable.

For me, that rules out anything that is exclusively CORS-like. Navigation is commonplace and adding an extra round trip to all navigations from an app isn't a great outcome, even if it is only for cross origin navigations (we don't need to add that barrier structurally, even if a lot of applications insist upon it).

However, it isn't quite that simple. The app lists what it thinks is in scope, so the point of this design is to confirm. We would start from an assumption that the app is making a correct representation, then confirm that with the app. That isn't CORS-like, that's something new (-ish, the introduction of Sec-Fetch-Dest is somewhat of a confirmation stage, as are some of the other CORS headers that determine whether a response is readable).

So I'm of two minds here. I hate the constant addition of stuff to HTTP requests. It's really starting to get out of hand (especially with very long field names, the risk that requests exceed the size of a packet is real and that has serious performance implications). But Reilly's suggestion has real merit. I also see the advantages of following that model of confirmation.

@dmurph's concern about two fields is easy to answer, I think: the site likely needs to know who is asking before they answer. Because the answer could depend. Sites could have resources that are acceptably included in multiple apps.

The manifest-like approach is also reasonable. A centralized location where you can interrogate an origin about the scopes that can extend to it is OK. I find myself horrified at the complexity of the proposals though. Please, if we go that way, can we focus on what is the minimum possible syntax that will achieve the desired outcome. This works:

{
   "https://example.com/this-is-an-app/": ["/payments", "/anti-fraud-stuff"],
}

(Yes, it is not extensible, but it is replaceable and that is enough.)

However, throwing out ideas is not what we need here. What is needed is to ask one question: What is the form of this that site operators are best able to handle?

Discussed Sep 1, 2024 (See Github)

No progress to report.

Discussed Sep 1, 2024 (See Github)

Martin: this was an early design review - they are going off and updating the spec. I think our involvement should end. We should ask them "when you're done, come back."

Dan: sounds reasonable to me.

Martin: because it's an early review...

Closed as "validated"

Comment by @dmurph Sep 4, 2024 (See Github)

From a chat with @LuHuangMSFT and a member of our security team, here is a proposal they were comfortable with. Not sure if this is something you're still OK with @LuHuangMSFT:

There are three levels of security for this association:

  1. None (not used)
  2. The .well-known/scope-extensions file acknowledgement on the extended origin/site.
    • If the origin/site doesn't have the manifest_id in that file, the association isn't created.
    • This can be re-checked periodically, allowing removal of associations but sometimes delayed.
  3. Both a .well-known/scope-extensions file acknowledgement on the extended origin/site AND a ScopeExtensionAllowed: <manifest_id_list> response headers coming from 'extended' sites or origins.
    • If the user agent ever does not receive this header from the server, then it removes the association. (Recoverable through existing manifest update mechanism if this was a dev mistake).
    • This allows a site/origin owner to immediately remove this association.

For origin type scope extensions, this requires level 2 security. For site type scope extensions, this requires level 3 security.

Reasoning:

  • None of these impede navigation / would affect performance.
  • The information is always known at install-time.
  • The origin case is more common for smaller developers & remains easy to do & set up - no need to muck with network neaders.
  • The site case is more complex & requires a more complex system that is likely a larger organization anyways, so setting that up to always include the correct response header seems OK.
  • The App-Id header seems not necessary - This only seems necessary if the site is expecting to have a very large list of apps extending to it, so it doesn't want to have a large network response. There have been no use-cases that we know about that require this or match this behavior. Also - this would be very hard to implement.
    • If needed in a future, then this can still be added.

Note this is separate from the filtering discussions, just for security here. Filtering can be handled by the current proposals with the .well-known/scope-extensions file and manifest file.

Comment by @LuHuangMSFT Sep 4, 2024 (See Github)

Thanks @dmurph. That's representative of what we discussed in that meeting. It is good solution that does not require a request header and I agree with the tiered approach for origin vs. site extension, as well as the reasoning bullet points.

App-Id is not necessary if the response header returns all manifest IDs that the origin recognizes. For completeness, I want to point out that not implementing an App-Id request header will not allow the server to control filtering differently for different origins. The filtering information (paths, URLPattern, etc.) will be in the association file and the same information will be applied to all origin matching a site extension.

Since

  • Per origin filtering when using a site extension is not a requirement for the known use cases, and
  • Nothing prevents App-Id from being specified in the future if the need arises, and
  • It would be easier for site operators to configure,

only requiring a response header with manifest IDs for site extension case is my preference.


Other issues

  • We can find a shorter name for the response header field (Extends-Apps?)
  • Re. specifying minimal filtering syntax - will replacement instead of extension not cause compatibility problems?
Comment by @martinthomson Sep 4, 2024 (See Github)

@dmurph that sounds encouraging. I'm a little unclear on the site vs. origin thing, but rather than continue the discussion here, can I suggest that you take this back to the explainer and update that?

Comment by @LuHuangMSFT Sep 4, 2024 (See Github)

I'll make an explainer PR that should offer more clarity.

Comment by @martinthomson Sep 11, 2024 (See Github)

Thanks for taking on the feedback. We understand that this is an early review and it will need some more work to integrate the changes. When you get things more settled, we'd be happy to do another view. Closing this for now.

Comment by @LuHuangMSFT Nov 14, 2024 (See Github)

@martinthomson I have updated the explainer [1] significantly to incorporate feedback and move the contentious portion involving the header design out of scope (this portion will be rewritten as a new, additional explainer).

Could we re-open this review?

[1] https://github.com/WICG/manifest-incubations/blob/gh-pages/scope_extensions-explainer.md