#482: URL Protocol Handler Registration for PWAs

Visit on Github.

Opened Mar 6, 2020

Hello TAG!

I'm requesting a TAG review of URL Protocol Handler Registration for PWAs

This feature will allow web developers to register web applications as URL protocol handlers via a property in the web app manifest. The apps would open directly when a custom-scheme URL is visited. Implementation-wise, we intend to build this feature by reusing registerProtocolHandler internally and modifying the protocol handler registry and shell integration classes to account for the existence of PWAs. The idea is to avoid code duplication and maintain the APIs aligned. (see explainer and design doc for more details)

Further details:

  • I have reviewed the TAG's API Design Principles
  • The group where the incubation/design work on this is being done (or is intended to be done in the future): WICG
  • Existing major pieces of multi-stakeholder review or discussion of this design: w3c/manifest#846 blink-dev WICG discourse thread
  • Major unresolved issues with or opposition to this design: None
  • This work is being funded by: Microsoft

We'd prefer the TAG provide feedback as:

💬 leave review feedback as a comment in this issue and @-notify @fabiorocha @maggers @connor-moody @samtan-msft

Discussions

Discussed Mar 1, 2020 (See Github)

Yves: Just looked at this, linked to issue we have on miniapps. It's basically something related how to define hooks based on on the URL scheme instead of doing it using media types. In that case... just skimmed it a bit earlier on ... seems that the ? mechanism is still http/https. I think there's something in webarch for that. What do they really want here? Do they just want media types, or do they want a protocol to access them.

David: I think one thing that's led people to want schemes like this in the past that they want the URL and not just the contents of the resource.

Tess: Why not just use registerProtocolHandler.

Yves: It's expensive, and (missed).

Dan: I kind of had the same question. Feels like web applications already do this. Doesn't feel like there should be a PWA-specific thing. The point of PWAs is that they're web applications; we can push back against technologies specific to PWAs, especially when web applications can already do this.

Tess: Restated: There could be some value in providing any web application that uses a web app manifest to have a way of saying what protocols they register from their manifest. But I'd want to think that's syntactic sugar, 1:1 with a call to registerProtocolHandler from the web application. Same underlying model and permissions and expectations. If we want to change the model for the PWA case, I'd want to entertain changing the model for the non PWA case. The manifest is convenience but not sole mechanism -- letting you know stuff that you could know anyway if you load the webapp -- it's letting you skip the loading. I worry that redefining the feature risks bifurcating the models. This should be by reference to registerProtocolHandler, or they should file bugs on registerProtocolHandler.

Tess: I'll take an action to review the explainer with that in mind and see if that feedback makes sense when I read it.

Dan: Bump it a week

Discussed Mar 1, 2020 (See Github)

Kenneth: looked at this and it seems sensible. Similar to register protocol handler but in the web app manifest... all can be done at "install" time. Same mitigation that exists in registerProtocolHandler. Not all protocols. Thinking about extending that a bit. Registering a scheme.. associated a title + and icon..

Dan: Tess said " could this be syntactic sugar on top of registerProtocolHandler"?

Ken: not totally – eg you can't associated an icon.

peter: you can have a tittle with registerProtocolHandler. I'd like to see the two things be more integrated. Maybe turn it to an options dictionary.

Ken: yes – also I'm worried that people will register a handler and use an icon and title that makes it look like another application. Something like that could be solved in the UI...

Yves: my comment last week was more about the use case section - e.g. for powerpoint. For magnet it's OK because it's a protocol to retrieve something. But web+powerpoint is a media type – there should be an option for what you encounter when you use a media type.

Ken: already another proposal for file extensions .. mime types vs file extensions discussion...

Peter: the thing about mime type is that you don't know what the content type is before you fetch it.

Yves: if it's in the link header you can ...

Ken: file handling proposal (#371). Instead of protocol handlers you have file handlers... so why are these not working the same way? Action thing instead of a URL. No icon associated with the file handler – should there be? Maybe these could just be "handlers" ...

Dan: feels like it could be confusing for the developer – different ways to do similar things.

Yves: if you're abusing media types and the way you're doing things based on content then it's another way of doing things differently from the current way of the web.

David: the file handling proposal is tied to the native filesystem API - lets the web page read and write files. Some concerns about native filesystem.

Ken: I still think the APIs could be similar.

Yves: official URI scheme – doesn't contain powerpoint, etc...

Peter: i noticed that registerProtocolHandler contains a list of safe schemes. That seems like something that should be harmonized or at least explained. Like to see some explanation of how this interacts with registerProtocolHandler.

Ken: also what happens if the manifest is updated and they add another one.

Peter: I'd also like to see the icon backported into registerProtocolHandler.

Ken: the UI should mitigate against misuse of this to confuse users.

David: that might involve something like showing the origin.

Peter: related topic: the list of safe listed schemes is being managed in the HTML spec... This should be an external registry.

Dan: isn't this encompassed in the evergreen /ever blue / ever teal / whatever color we are using proposal?

David: living standards have some of the advantages of registries too.

Rossen: will summarize on the github

Comment by @torgo Mar 11, 2020 (See Github)

@samtan-msft thanks for sending this. Do you have an idea where this might go after incubation in WICG?

Comment by @fabiorocha Mar 18, 2020 (See Github)

@torgo we initially thought about having it incubated in WICG, but as discussed on this thread the spec side of this work might be rather small and it has been suggested it could be treated as a spec bug and discussed as an issue on the manifest repo. What do you think?

@marcoscaceres how does that sound to you? We already have w3c/manifest#846, and we could just iterate there?

Comment by @samtan-msft Apr 29, 2020 (See Github)

Hey, quick question, is anyone looking at this on the TAG side at this moment?

Comment by @marcoscaceres Apr 30, 2020 (See Github)

@marcoscaceres how does that sound to you? We already have w3c/manifest#846, and we could just iterate there?

Sorry I missed this earlier. That seems fine... however, it doesn't hurt to get TAG feedback :)

Discussed May 1, 2020 (See Github)

Tess: i wonder about the security implications of the manifest and registerProtocolHandler being out of sync...

Tess: Seems like they allow the PWA to handle https too, which is worrisome.

Dan: i left a comment and pushed to f2f

Comment by @torgo May 11, 2020 (See Github)

We are re-evaluating based on the new info in w3c/manifest#846... @marcoscaceres do you have any additional thoughts on this you could share in the context of this review? One thing we're concerned about is the possibility for things to get "out of sync."

Comment by @hober May 26, 2020 (See Github)

Hi @fabiorocha @Maggers @connor-moody @samtan-msft!

@kenchris and I looked at this during a breakout of this week's TAG F2F. Overall we like the idea of adding this functionality to Web App Manifest, but we share the concerns our colleague @dbaron raised in https://github.com/mozilla/standards-positions/issues/340#issuecomment-631797523. We'd like to see this new manifest feature and the existing registerProtocolHandler() specced on top of a common underlying model, like how the Fetch spec unified the model for loading subresources. That way we could all trust that these things wouldn't get out-of-sync, and that they could have consistent UIs, permissions, etc. So ideally you'd have a pair of pull requests, one on Web App Manifest and one on HTML, that refactored the existing HTML text to provide the underlying model, and then exposed an additional API into that model in Web App Manifest.

Comment by @hober Sep 22, 2020 (See Github)

Hi @fabiorocha @Maggers @connor-moody @samtan-msft,

Any thoughts on the above?

Comment by @fabiorocha Sep 23, 2020 (See Github)

@hober My sincere apologies for letting this fall through the cracks. Thanks for the ping!

I think that makes a lot of sense, conceptually. Would you have pointers to the Fetch changes you mentioned as an example for this? Do you think both changes need to be staged at the same time or can one come before the other? I'd appreciate further guidance here.

Discussed Oct 1, 2020 (See Github)

Peter: marked as pending feedback and looks like we got some...

Tess: ... kind of "the entire thing" - I'll reply.

Peter: going to waiting for feedback?

Comment by @hober Oct 12, 2020 (See Github)

@hober My sincere apologies for letting this fall through the cracks. Thanks for the ping!

I think that makes a lot of sense, conceptually. Would you have pointers to the Fetch changes you mentioned as an example for this?

The addition of the Fetch spec itself is the change I was referring to. Before Fetch, we had XHR. When Fetch got specced, a common model was defined that both XHR and the Fetch API could be defined on top of.

Do you think both changes need to be staged at the same time or can one come before the other?

I think it would be best if they were both staged at the same time; that way, each can link to the other so people understand why the changes are happening. If one came before the other, it should probably be the HTML change that happens first.

Discussed Jan 1, 2021 (See Github)

Requested an update...

Comment by @torgo Jan 26, 2021 (See Github)

Hi @fabiorocha - @kenchris and I are just coming back to this issue in our "f2f" today and we're wondering if there has been any update? Can you let us know the current status? Is this something we should close for now?

Comment by @diekus Mar 8, 2021 (See Github)

Hola TAG,

We are looking into this. We opted for not changing the HTML5 spec and are rather thinking of making the Manifest spec dependent on the HTML one by referencing parts of it. We are working on the PR and will update with more info on this very soon.

gracias

Comment by @domenic Mar 8, 2021 (See Github)

Hey, HTML spec editor here. I don't think it's appropriate to change how navigation works without changing the HTML spec... See also https://github.com/w3ctag/design-principles/issues/184 and https://annevankesteren.nl/2014/02/monkey-patch .

Comment by @aarongustafson Mar 9, 2021 (See Github)

Hey, HTML spec editor here. I don't think it's appropriate to change how navigation works without changing the HTML spec... See also w3ctag/design-principles#184 and https://annevankesteren.nl/2014/02/monkey-patch.

@domenic I don’t believe we are monkey patching. Our current plan—which @diekus is referring to—is to reference the HTML spec’s logic for normalizing/vetting protocol handlers, which is analogous to how ImageResource references HTML’s sizes parsing and how the Manifest currently references ImageResource processing and the Screen Orientation API. The reason for the reference (as opposed to defining a bespoke mechanism) is that any protocols that get added to (or removed from) HTML spec will get picked up automatically.

Assuming HTML doesn’t abandon the Protocol Handler API, we don’t foresee it becoming an issue. And if it comes to pass that registerProtocolHandler() becomes deprecated or is otherwise removed from the spec, we’ll resolve the ReSpec issue by bringing the referenced normalization logic into the Manifest spec.

As I understand it, the plumbing of the two approaches will be largely the same as this is, essentially, a declarative means of doing what registerProtocolHandler currently does programmatically (@fabiorocha and @diekus can get into the specifics on that, if needed); the only difference will be routing users to the installed PWA as opposed to the browser itself. As it is an installed PWA-specific behavior, it feels like that that should be defined in the Manifest spec, as opposed to HTML. Or are you thinking that we should include the navigation resolution (PWA vs. browser) in HTML so that calls to registerProtocolHandler, if made from within the context of an installed PWA, could use it as well (and we would just reference that from the Manifest)?

Comment by @domenic Mar 9, 2021 (See Github)

Hmm, maybe I misunderstood this then. Sure, definitely reference HTML's logic for normalizing/vetting protocol handlers.

But what spec will be modified to actually say that when a manifest registers for web+music://, navigations to web+music:// must launch the PWA? My point is that specifying that requires changes to HTML. If this were instead specified in the manifest spec, then that would be a monkeypatch of the problematic kind discussed above.

Or are you thinking that we should include the navigation resolution (PWA vs. browser) in HTML so that calls to registerProtocolHandler, if made from within the context of an installed PWA, could use it as well (and we would just reference that from the Manifest)?

Indeed, navigation is defined in HTML. However this is orthogonal to registerProtocolHandler. Any navigations that are intercepted, regardless of whether they're intercepted because of registerProtocolHandler or because of some manifest entry, are handled by HTML.

Comment by @aarongustafson Mar 9, 2021 (See Github)

@domenic Thinking about verbiage, how would you frame the state of being installed? Is that considered/mentioned elsewhere in the HTML spec?

Comment by @domenic Mar 9, 2021 (See Github)

So far I am not aware of any web platform features which work differently in installed contexts vs. not. (That might be a good architectural issue for the TAG to consider for this review!)

If you did want to proceed down that path, introducing such a definition (either in the manifest spec or the HTML spec) which HTML could reference would make sense.

Comment by @marcoscaceres Mar 18, 2021 (See Github)

So far I am not aware of any web platform features which work differently in installed contexts vs. not. (That might be a good architectural issue for the TAG to consider for this review!)

Let's hope we don't start getting that kind of divergence. That would not be great.

If you did want to proceed down that path, introducing such a definition (either in the manifest spec or the HTML spec) which HTML could reference would make sense.

Agree. Though I'm hopeful that apart from very subtle/transparent things (e.g., maybe an increased storage quota, or slightly longer cookie persistence), nothing will differentiate installed from regular web apps.

Discussed May 1, 2021 (See Github)

Ken: I know that Diego was involved with this issue but I don't see him as part of the primary contacts. I will reach out to him and ask if he can be added.

Dan: I agree with Marcus' statement about "apart from subtle things nothing else will be different".

[discussion on whether you need this or if register protocol handler is enough?]

Rossen: would the same be applicable in app to app scenarios? More applicable example - i click on an email, my handler has outlook PWA registered. in outlook i should be able to click ...

Dan: Or example where you have email client running in a Tab.. and you want clickin on an email link to bring you to that tab and open up a compose function there... Can register protocol handler do that?

Ken: Yes.

Ken: I think it's the same... it just calls that function underneath. So that it gets registered when you install the app. That's fine - it's not something you want every site to do, but when you install something it could be a good time to ask.

Dan: so it's fine - as long as regular web apps running in other tabs for example also have access to the same functionality.

Ken: yes and that seems to be the case.

Rossen: I think that if Diego or Aaron engages it will move this along.

Comment by @kenchris May 12, 2021 (See Github)

@diekus maybe you want to add your self to primary contacts and give us an update on the status of this

Btw this was published yesterday: https://web.dev/url-protocol-handler/

Comment by @diekus May 12, 2021 (See Github)

@kenchris Hola Kenny ! How do I add myself as "primary contact"? I am the PM for this feature so indeed I am your preferred point of contact. (Sorry my first time helping with a TAG review).

Thanks for the heads up about the web.dev article, our team at MS collaborated with this indeed!

To give you an update about this, we have submitted 2 changes to the HTML spec to add 2 new definitions to it to align the manifest spec with the processes in the registration and invocation of protocol handlers in the HTML one. You can see the PRs here and here. They have both been merged and I have unblocked this to continue work on the manifest spec.

Comment by @kenchris May 12, 2021 (See Github)

@diekus I added you now

Comment by @atanassov May 12, 2021 (See Github)

@diekus thank you for the quick reply and detailed update. Looking how far this has matured it looks like we should be able to sign off unless there is anything else in your mind that we need to go over?

Comment by @diekus May 12, 2021 (See Github)

@atanassov All is good on our side and I'm happy to sign off. Just out of curiosity, in the TAG review process, what does this sign off mean? is it like an A+ on your grades?

Comment by @kenchris May 12, 2021 (See Github)

We (@torgo, me and @atanassov) looked at this in our breakout today and we are happy with the progress and the collaboration with WHATWG on getting this aligned with existing specs and getting the changes incorporated into HTML.

Going to close this! Thanks for bringing this to the TAG!