3. We're concerned that this proposal doesn't talk about providing equivalents for the text that is placed on the canvas that would be exposed to assistive technologies. One option the platform offers is canvas fallback content, though it has its limitations. Have you considered how this barrier may be overcome, and/or what specific advice should be given to content authors?

(Commenting as a colleague of Stephen's who provided accessibility guidance during the course of this work.)

My understanding is that the current state of the art is (unfortunately) for canvas-based applications to implement self-voicing solutions based on aria-live regions. I believe Stephen is investigating the extent to which fallback content is even used in practice for accessibility-related purposes, but what I hear from various sources is that canvas fallback content isn't really fit for purpose for most production use cases.

I think this represents a serious gap in the platform, and I agree that adding APIs to support more sophisticated use cases for text in canvas does make the need for addressing this gap increasingly pressing. However, I'm not sure what guidance can usefully be given in the context of this API while that gap persists.

Thanks for chiming in, @alice!

A wild thought, but what if these text rendering functions took references to elements in the fallback content instead of plain text strings? They'd probably need to be phrased as callbacks to handle when the text of the elements changes, and we could think of them as special cases for rendering what sighted people need, instead of making the text-only people be the special case. Doing that might also motivate non-accessibility developers to fix the problems with fallback content. And it could allow the full range of multilingual text in those elements instead of assuming the whole string is the canvas's language.

Ah, taking elements and rendering them in canvas. That's exactly what we're working on with HTML-in-Canvas in an effort to improve accessibility for content best representing by DOM. It raises it's own set of a11y concerns, but in general should make it easier for authors to match their canvas content with their fallback content as you suggest.

In this case we're building on existing canvas text features, and don't want to mess with the API too much due to existing web content.

My collaborator in this work, @fmalita, is looking into the functional questions you raised. I take the point about ligatures and writing modes.

For getIndexFromOffset() (Point 2), canvas has the notion of LTR and RTL but nothing beyond that. The assumption is that an author will use transforms to align text vertically. There is also the problem of mimicking sideways-* which IIRC needs to rotate individual characters in some situations, which I think you would need this proposal to do correctly. Regardless, to get the correct position to use for getIndexFromOffset I believe an author would need to transform the point into the coordinate system of the text string to correctly compute the offset along the string's rendered orientation.

Ligature breaking is possible and I agree should happen, so we will define the correct behavior in any spec text and update the explainer if necessary. For italics and the like we would behave the same as the DOM caretPositionFromPoint method which does not take into account italics. The underlying implementation in browsers uses the editing code paths (also for selection) so the behavior would match what those systems do. In Chromium, at least, the selection rect does not cover italic characters in all cases.

The copying of align and baseline (Point 4) is done to simplify the rendering when measurement and creation of the glyph clusters is done in a different code location. That is, if you create in one function then pass around to another or store for later. It captures the alignment and baseline at measurement time so they can be re-applied at rendering time. I believe this came out of the spec PR discussion. Note Florin and myself have taken over this work from @AndresRPerez12.

Thanks for reviewing @jyasskin!

• The example code passes end=text.length, which is the number of code units, but the algorithm throws an exception if it exceeds the number of code points. Since the emoji each take 2 code units to represent a code point, this should throw. However, Intl.Segmenter operates in terms of code units, and it's important to be consistent with that here, so you should probably update your algorithm to use code units too.

Good catch, that looks like an unintended discrepancy. Agreed that consistency with Intl.Segmenter is desirable, plus working with code points would introduce some extra client friction.

We should update the spec PR to use code units.

Interestingly, the Chromium prototype impl is already using code units, thus the example doesn't throw. You can see it running live here if you launch Chromium with --enable-experimental-web-platform-features.

(The example does require a minor tweak to match the current spec PR: TextCluster.begin has been renamed to TextCluster.start.)

• We see that the "fi" ligature is getting a single color. We suspect that counts as "their corresponding glyphs cannot be broken down any further" in your spec.

Indeed, the focus is on glyph clusters (as produced by the text shaping engine/text preparation algorithm) that cannot be subdivided any further without breaking the user-perceived "character". They roughly correspond to grapheme clusters, but they also include ligatures because those are single glyphs that cannot be subdivided (I've heard some of our font folks refer to them as "glyphemes").

One thing we should probably emphasize in these docs is that TextClusters are constructed in the glyph (post-shaping) domain, and are indirectly related to the Unicode domain.

1. The explainer refers to these TextClusters as "grapheme clusters" in a couple places, but "Grapheme clusters are not the same as ligatures."

Point taken, some of the explainers are using loose (or possibly outdated) terminology - we should update them. Note that the spec PR avoids any mention of graphemes and sticks to the accurate description.

2. One of your use cases for this API is for caret positioning, but the caret has to be able to be positioned between the "f" and "i" in that ligature.

Since ligatures are single glyphs, there is no way to get the partial font metrics required for accurate mid-ligature positioning. AFAIK most editors don't support mid-ligature positioning, and the ones that do rely on approximations/heuristics.

At a high level, this seems to fall onto clients: if caret positioning is important, they should use fonts without ligatures, or disable ligatures during text shaping. (Whether that latter part is achievable with Canvas is a good question... AFAICT, unlike CSS, there is no way to explicitly disable ligatures with Canvas APIs. I found this old proposal that could be extended for font-variant-ligatures, but that's a separate effort. @schenney-chromium)

I think this represents a serious gap in the platform, and I agree that adding APIs to support more sophisticated use cases for text in canvas does make the need for addressing this gap increasingly pressing. However, I'm not sure what guidance can usefully be given in the context of this API while that gap persists.

Hi all. Mike here, member of AGWG and APA but my comments are purely my own. New poster here in this thread and just curious about a couple of things:

1. @schenney-chromium Have you discovered anything about that gap that Alice mentioned and if there's any development? It doesn't seem like there's a workaround at the moment, but thought I'd ask anyway; and

2. With the ability to customize kerning, skew, orientation, etc., also comes the chance for rendered text in a typically left-to-right language to be presented right-to-left. This is true of the example on the project page that renders the text in a circle, which has upside-down and right-to-left text presentation. I'm unaware of any way to solve for this customized rendering except for having plain text available within a close proximity (reading order) to the rendered text, or relying on canvas fallbacks (which are limited). I'm trying to think about other methods to ensure as accessible an experience as possible. Within the proposal, is there room to have an automatic output that's rendered in real text adjacent to the canvas? Thinking out loud here...