#1043: CSS.highlights.highlightsFromPoint API

Visit on Github.

Opened Jan 22, 2025

こんにちは TAG-さん!

I'm requesting a TAG review of highlightsFromPoint API.

The highlightsFromPoint() API is designed to enhance user interaction with highlighted content on web pages. Highlights are used to visually distinguish specific ranges within a document, such as comments, search results, or spelling errors. This API aims to provide a robust mechanism for identifying and interacting with these highlights, addressing challenges related to multiple overlapping highlights, shadow DOM handling, and performance optimization. More details were discussed in the original issue filed in csswg: https://github.com/w3c/csswg-drafts/issues/7513.

Further details:

  • I have reviewed the TAG's Web Platform Design Principles
  • Previous early design review, if any: N/A
  • Relevant time constraints or deadlines: None
  • The group where the incubation/design work on this is being done (or is intended to be done in the future): CSSWG
  • The group where standardization of this work is intended to be done ("unknown" if not known): CSSWG
  • Major unresolved issues with or opposition to this design: None
  • This work is being funded by: Microsoft

Discussions

Log in to see TAG-private discussions.

Discussed Feb 1, 2025 (See Github)

Xiaocheng: this feature is an extention to custom highlight API. Provides a JS API takes coordinates and returns custom highlight. My opinion is "unsatisfied" because this API requires a synchronous... easily misused and cause performance problems... Their use case for this API when user performs interaction... with highlight, they want to modify the highlight... they want to know when and which highlight the user is working out. This API is not the correct solution to their intended use case. Better model would be using events or callbacks.

Dan: is there something like "alternatives discussed"?

Xiaochen: no they didn't consider...

Dan: I think the privacy considerations seems light.

Xiaocheng: I think the privacy concerns are OK in the spec itself .. the API only covers custom highlights... it only identifies [that kind of highlight]... Also the structure doesn't contain document info...

Dan: that makes sense...

<blockquote> Hi @ffiori - thanks for sending this our way. We discussed in a TAG breakout meeting this morning. One question we had was regarding the choice to make this a synchronous API vs using something like an event model. We are concerned about the performance impact of using a synchronous approach. Have you considered asynchronous alternatives like firing an event when user interacts with a custom highlight? </blockquote>

Matthew: raises potential a11y issues we (APA) encourage people to put a a11y considerations section... I might suggest that. They've thought about it.

matthew to leave comment

Discussed Feb 1, 2025 (See Github)

xiaocheng: last time we raised a concern ... we asked if they considered an event based API. They said they did but it's a deviation from existing event model... not any node in the DOM tree. So they went for simplest approach.. so I suggest close it with satisfied with concerns. We already have a class of layout APIs that are widely used...

... maybe we add a topic to the F2F?

Martin: in terms of layout thrashing .. I notice this not a promise based API either. Expectation is that the browser synchronously forms the API... It seems on the edge [of concerning]. You probably do want synchronous access... so maybe it's the right thing to do...?

Xiaocheng: it's like a trade-off... One one hand it's easy to use, on the other hand it causes thrashing... I've seen a lot of examples of similar APIs causing thrashing.

Martin: if you do this every time you request an animation frame for example.

Xiaocheng: doing something in scrolleventlistener and then requestanimatioframe... and then in intersectionobserver...

Dan: sounds like a 'foot gun'... Is there something we can encourage them to say in their implementation guidance?

Xiaocheng: introducing async alternatives.. querying a layout and getting a promise that resolves in next rendering cycle... By the way, I have seen this implemented in something called workerdom - has been discontinued.

Martin: i like that idea... a promise and it resolves when the next layout occurs... Why is that not an alternative?

Xiaocheng: i'm not aware ...

Hadley: we don't need to come up with the solution ourselves, but we can post the problem... Or put some guidance in the spec to help implementers find their own solution...

<blockquote> We're still concerned about UI thrashing... And the synchronous nature of this API could under certain circumstances could </blockquote>

Xiaocheng: this is a blind spot in the Design principles. Syncronous APIs should not have a long exclusion time. Async APIs, instead. This one needs a long execution time but on the same thread.

Comment by @xiaochengh Feb 12, 2025 (See Github)

Hi @ffiori - thanks for sending this our way. We discussed in a TAG breakout meeting this morning. One question we had was regarding the choice to make this a synchronous API vs using something like an event model. We are concerned about the performance impact of using a synchronous approach. Have you considered asynchronous alternatives like firing an event when user interacts with a custom highlight?

Comment by @ffiori Feb 12, 2025 (See Github)

Hi @xiaochengh, thanks for discussing the topic.

The reason for implementing highlightsFromPoint has been discussed with the CSSWG here before. Just wanted to make clear that by implementing this API we're not discarding implementing an event-based alternative in the future if problems arise or we find other use cases that could apply to that specific scenario.

Summarizing, we chose to move forward with highlightsFromPoint because it satisfies various use cases (for example, interactions with custom spellcheck, annotation popups over words, interactions with custom find-on-page, to name a few), and it's a much simpler approach than defining a whole new model for events (as was discussed in the same issue linked above).

Let me know if you'd like me to expand on some parts of my comment or if there's any other questions.

Comment by @xiaochengh Feb 13, 2025 (See Github)

@ffiori Thanks for the prompt reply!

I see this hit-testing API has been compared with an event API, and major reason the CSSWG moved away from it due to its much higher complexity. Is that correct, and did I miss any other points?

Btw it will be great if the event API is listed in the Alternative Solutions section of the explainer with a comparison.

Comment by @ffiori Feb 14, 2025 (See Github)

Hi @xiaochengh, thanks for the quick reply. You're right about the reason for choosing this API over the event API.

I just updated the explainer as suggested. Let me know if you have any other questions or concerns.

Comment by @xiaochengh Feb 19, 2025 (See Github)

Hi @ffiori, the TAG discussed it in a breakout today and are still concerned with adding an API that may block the main thread and cause layout thrashing. Per Design Principles, synchronous APIs should have short execution time. Are there other asynchronous alternatives that have been considered?

Comment by @ffiori Feb 26, 2025 (See Github)

Hi @xiaochengh, thanks for following up on this topic by filing an issue. One of the reasons why we found this API to be worth pursuing is because it maintains the current patterns for developers (like other *FromPoint APIs, i.e. elementFromPoint, caretPositionFromPoint) so it's easier and intuitive for them to adopt it.

It's also up to the developers to call the API with a clean layout. We can give them some guidance by adding a note in the spec and in MDN mentioning this. We can also give an example like the following:

document.addEventListener('click', (event) => {
    requestAnimationFrame(() => {
        requestAnimationFrame(() => {
            const highlights = CSS.highlights.highlightsFromPoint(event.clientX, event.clientY);
            // Do something with the highlights
            console.log(highlights);
        });
    });
});
Discussed Mar 1, 2025 (See Github)

Xiaocheng: last time, our major concern is that this might cause layout thrashing... we asked if they explore async.. Their response is "we can use this function properly so it doesn't cause layout thrashing" ... which I don't agree. No way to use this function 'properly' to completely avoid layout thrashing... I feel we've done a lot of rounds ... they have insisted on this approach... I think (not confirmed) their major positions: 1. there are already a class of functions like this so it's not a big deal and 2. this is the simplest solution to the use case they are trying to solve. So I'm wondering what our response should be. I feel like we should close it. Is it satisfied, with concerns or even stronger?

Hadley: how consequential is layout thrashing?

Martin: some people don't think it's a problem but those would be in the minority in this ... discussion.

Matthew: personally I'm sensitive to layout thrashing... Seems like something we would like to avoid if we could. I do understand they want to maintain consistency. But the user is the top priority. what do other engines say?

Hadley: standards positions?

Xiaocheng: filed but no positions yet...

Martin: I think it's up to us to make a decision ... if sync APIs that perform layout are a good idea for the platform... The PR https://github.com/w3ctag/design-principles/pull/563 I suggest we point to?... if there is a bad example, don't use that to justify another thing that replicates that bad example.

Hadley: it sounds like we're not enthusiastic about it... If we've continued to engage and they are not taking into account our concerns then I'd lean towards "unsatisfied". We want to make it clear to the w3c community what we think about this... We ran into trouble with this on DIDs... we weren't clear about our feedback / concern...

Dan: I think we should Martin's phrase, the general issue about synchronous functions that are involved in layout. I think that gives clarity.

Martin: our current position on that needs some work in the design principles.

  1. Synchronous APIs are bad, wasteful, cause performance problems, and annoy users with flashing (see https://w3ctag.github.io/design-principles/#synchronous)
  2. We can't use "we have existing synchronous APIs" and therefore it's okay to add this — because of our "leave the web better than you found it" design principle.

Action: Xiaocheng to draft comment and share it asynchronously -- aiming towards closing as unsastisfied

Comment by @xiaochengh Apr 7, 2025 (See Github)

Hi @ffiori, the TAG discussed it and still found our concerns unresolved. We understand that this API is the simplest solution to the target use case, but we are concerned with synchronous layout-performing APIs in general, and the introduction of another API in this class might not be the good way to go.

  1. Synchronous layout-performing APIs, on their own, violates design principle be synchronous when appropriate:

An API should generally be synchronous if the following rules of thumb apply:

  • ...
  • The execution time is short and deterministic.

  1. There's no way to reliably prevent layout thrashing caused by such APIs, as otherwise we wouldn't have invented Intersection Observer.

  2. Consistency with existing layout-performing APIs does not justify the introduction of another one. We should leave the web better than you found it, and in particular:

Issues that are present with a certain web technology now may be fixed in a subsequent iteration. Duplicating these issues makes fixing them more difficult. By adhering to this principle we can make sure overall platform quality improves over time.

For now we are closing this review. We still encourage the exploration of asynchronous alternatives, be it events, observers, Promises, ...

Comment by @zcorpan Apr 24, 2025 (See Github)

I commented in https://github.com/mozilla/standards-positions/issues/1068#issuecomment-2788724258

I think there is value in consistency with the existing *FromPoint APIs (which also flush layout).

If we want to provide a mechanism to use APIs at appropriate times without flushing layout, I think that should be done holistically for all layout-flushing APIs.