#980: Multiple import maps
Discussions
Comment by @LeaVerou Aug 9, 2024 (See Github)
This is very exciting stuff! 😍 1
As a very initial comment, could you please add usage examples to the explainer (or a move it to actual, separate explainer) that show what design is actually being proposed? We simply don't have the bandwidth to be wading through pull request diffs to hunt this information down, and I really wouldn't want something like this to fall off our radar because of this.
Footnotes
-
A comment to the problem being solved; I have not seen the proposed design because looking for it exceeded the time I had to allocate… ↩
Comment by @yoavweiss Aug 9, 2024 (See Github)
Thanks!!
I added a usage examples section to the explainer.
We simply don't have the bandwidth to be wading through pull request diffs to hunt this information down
Yup, totally understandable! Let me know if the above helps, or if you'd like me to pull the explainer to its own doc. (The current form felt more "discoverable", but happy to move it if that helps)
Discussed
Oct 1, 2024 (See Github)
We looked at this today, and it seems like a very nice improvement to the platform. The devil will be in the details, of course. It looks like you're working those out in the right place. Please keep doing that, and good luck.
</blockquote>Then a question:
If querying the map causes the queried mapping to be fixed (otherwise, the introduction of a new mapping will result in a change to the already-retrieved mapping) then does that not potentially lead to non-deterministic outcomes? The value of mappings will depend on the order of import map availability and queries to the import map. This is partly addressed by the fact that a module graph uses a consistent mapping, but that doesn't help at the top level where the loading order of modules might not be deterministic. Also, why should a miss (a mapping that a module graph attempts to resolve, but fails) be treated differently, as it appears to be?
Jeffrey: This and https://github.com/w3c/tpac2024-breakouts/issues/31 / https://github.com/WICG/webcomponents/blob/gh-pages/proposals/css-modules-v1-explainer.md seem related, unless I'm mixing up module maps and import maps.
<blockquote>At first glance, this looks great. We had a few questions:
- As we understand it, the requirement to ensure that a mapping cannot change is hard to meet. It looks like a resolved mapping has the effect of fixing that mapping so that new additions to the map don't change it. How do you avoid nondeterminism? We see that you require the mapping to be stable while loading a module graph, but what about between different graphs? We're thinking about top-level async module loading here and we're not clear on how this might interact with that.
- If a mapping is used, but does not resolve successfully (perhaps because no mapping exists) is that a state that needs to persist? Why is that safe?
- The discussion in https://github.com/w3c/tpac2024-breakouts/issues/31#issuecomment-2387141476 seemed to indicate that server-rendered custom elements would want to add inlined CSS modules to the module map as they're sent to the client. Does that need support from this proposal, and if so does this proposal support it? cc/ @justinfagnani @rniwa
Discussed
Oct 1, 2024 (See Github)
satisfied
:
Thank you for the discussion. We're happy to see this proceed through the HTML review process.
</blockquote>
Discussed
Oct 1, 2024 (See Github)
Closed after previously saying we're OK.
Comment by @martinthomson Oct 15, 2024 (See Github)
@yoavweiss, at first glance, this looks great. We had a few questions:
-
As we understand it, the requirement to ensure that a mapping cannot change is hard to meet. It looks like a resolved mapping has the effect of fixing that mapping so that new additions to the map don't change it. How do you avoid nondeterminism? We see that you require the mapping to be stable while loading a module graph, but what about between different graphs? We're thinking about top-level async module loading here and we're not clear on how this might interact with that.
-
If a mapping is used, but does not resolve successfully (perhaps because no mapping exists) is that a state that needs to persist? Why is that safe?
-
The discussion in https://github.com/w3c/tpac2024-breakouts/issues/31#issuecomment-2387141476 seemed to indicate that server-rendered custom elements would want to add inlined CSS modules to the module map as they're sent to the client. Does that need support from this proposal, and if so does this proposal support it? cc/ @justinfagnani @rniwa
Comment by @justinfagnani Oct 16, 2024 (See Github)
Does that need support from this proposal, and if so does this proposal support it
I don't know if it needs support from this proposal directly, but I do think that the edge cases we talked about wrt inlined style modules that populate the module map are the same and should be handled consistently - probably by referring to common steps?
The main edge cases are race conditions between an import from a module in the module graph, and the handling of an inline module (or dynamic import map here), ie a fetch of a module is in-flight while an inline entry for the module is discovered. Assuming this proposal specifies what happens in those cases, it'll be a big help for the inline modules proposal.
Comment by @yoavweiss Oct 16, 2024 (See Github)
- We see that you require the mapping to be stable while loading a module graph, but what about between different graphs?
Apologies! That part has changed as part of the PR review, and I failed to update the relevant part in the explainer. After lengthy discussions, @hiroshige-g convinced me that a single global tree is better.
I agree that we can't get rid of non-determinism here entirely. As you say, async top-level modules where the import map is defined later may resolve their dependencies either before or after the map is processed. The same can be true for dynamic imports.
At the same time, I wouldn't expect this to be a problem in practice:
- Non-determinism already exists, and it doesn't seem to be a problem in practice (e.g. a dynamic import can invalidate an import map if it currently loads before it)
- I'd expect developers to define the import maps relevant to their modules and load them when needed. I'd wouldn't expect these maps to have a lot of conflicts in practice.
- A real-life scenario we're seeing is that different parts of an app are handled by different teams/entities, each managing its own modules and their mapping. In this case I'd expect zero overlap.
- Another real-life scenario this would enable is a large app that dynamically loads parts as they are needed, and injects the relevant mapping before the dynamic import is called. In these apps, any overlap between the maps is likely to be duplicate rules, and hence ignoring them won't change resolution.
- If a mapping is used, but does not resolve successfully (perhaps because no mapping exists) is that a state that needs to persist? Why is that safe?
Can you expand on this? I'm not following..
- The discussion in Sharing styles with Declarative Shadow DOM w3c/tpac2024-breakouts#31 (comment) seemed to indicate that server-rendered custom elements would want to add inlined CSS modules to the module map as they're sent to the client. Does that need support from this proposal, and if so does this proposal support it? cc/ @justinfagnani @rniwa
Injecting inline modules into the module map seems largely orthogonal to import maps. I played around with it a bit (as a prototype for some interesting loading experiments that I need to get back to), and I think the main question there is "how do we determine the specifier?". Once we've answered that, I don't know that it makes sense to use import maps to translate that specifier to a URL, where there is no such URL in practice.
I don't know if it needs support from this proposal directly, but I do think that the edge cases we talked about wrt inlined style modules that populate the module map are the same and should be handled consistently - probably by referring to common steps?
The main edge cases are race conditions between an import from a module in the module graph, and the handling of an inline module (or dynamic import map here), ie a fetch of a module is in-flight while an inline entry for the module is discovered. Assuming this proposal specifies what happens in those cases, it'll be a big help for the inline modules proposal.
While there are some similarities, relying on a specifier to be in the module map for resolution and defining specifier mapping in the import map seem like different problems, at least at first glance. For the latter, I think we can safely ignore rules which overlap with past import maps or past resolved modules. For the former, we'd need to decide what we do when the required modules are not (yet?) there at resolution time.
Comment by @martinthomson Oct 16, 2024 (See Github)
OK, with respect to non-determinism, it would be good to have some text on that in the specification. Or even some way to manage it (can a script that performs dynamic loading test whether a particular import map has been seen?).
The spec PR seems to be a little out of sync with what you just said (it's written in the usual impenetrable language, so I'm not 100% here) with what you say. It says something about the map being fixed and then propagated throughout each top-level module load.
If a mapping is used, but does not resolve successfully (perhaps because no mapping exists) is that a state that needs to persist? Why is that safe?
I think that you indirectly answered this one with your non-determinism answer. My question, expanded:
It appears as though there is a requirement that the map, once queried, cannot change. Does that apply only in cases where the query results in a successful resolution, or does it apply in other cases:
- An attempt is made to load "foo" and "foo" doesn't exist in the map. Does that mean that a future addition to the map cannot add a mapping for "foo"?
- An attempt is made to load "foo" and "foo" exists in the map, but the resolution fails (an HTTP 404 error on the indicated resource, say). Does that make "foo" available for overriding in a future addition to the map?
These are not particularly obvious to me from the explanatory material, though I'll admit to not having gone through all of your updated examples.
Comment by @yoavweiss Oct 16, 2024 (See Github)
The spec PR seems to be a little out of sync with what you just said (it's written in the usual impenetrable language, so I'm not 100% here) with what you say. It says something about the map being fixed and then propagated throughout each top-level module load.
Good catch. I removed that processing model, but left in an inaccurate note. I'll fix that.
If a mapping is used, but does not resolve successfully (perhaps because no mapping exists) is that a state that needs to persist? Why is that safe?
I think that you indirectly answered this one with your non-determinism answer. My question, expanded:
It appears as though there is a requirement that the map, once queried, cannot change. Does that apply only in cases where the query results in a successful resolution, or does it apply in other cases:
- An attempt is made to load "foo" and "foo" doesn't exist in the map. Does that mean that a future addition to the map cannot add a mapping for "foo"?
- An attempt is made to load "foo" and "foo" exists in the map, but the resolution fails (an HTTP 404 error on the indicated resource, say). Does that make "foo" available for overriding in a future addition to the map?
These are not particularly obvious to me from the explanatory material, though I'll admit to not having gone through all of your updated examples.
Good points! The addition of a module to the resolved module set happens when the module is resolved, but before any import map rules are applied. So you're right, and failed resolution attempts would end up preventing future rules from being set for these specifiers.
That doesn't seem like what we'd want here. I'll change the addition to only happen after a successful import map resolution (if any).
At the same time, for 404s, I don't think we should change future resolution for them, as they were properly resolved.
Comment by @martinthomson Oct 16, 2024 (See Github)
Glad to be of assistance. Do you have any thoughts on providing assistance with managing any non-determinism? I missed a step above, but the idea is that if a script performing a dynamic load depends on the presence of a specific import map, how would it guarantee that the map has been seen and integrated?
It seems like your resolution here would ensure that a missing mapping only gets committed once it is resolved (whether it fails or not), but that isn't ideal because a missing mapping would then permanently enter that state if the dynamic load acted before the mapping was in place. "Don't do that" is a perfectly fine response if there are obvious ways to manage that.
Comment by @yoavweiss Oct 16, 2024 (See Github)
Glad to be of assistance. Do you have any thoughts on providing assistance with managing any non-determinism? I missed a step above, but the idea is that if a script performing a dynamic load depends on the presence of a specific import map, how would it guarantee that the map has been seen and integrated?
Given that import maps are loaded synchronously (inline) that doesn't seem hard to manage from a developer perspective, as long as they control both pieces of code. But this is something we can add in the future as a function of developer demand.
It seems like your resolution here would ensure that a missing mapping only gets committed once it is resolved (whether it fails or not)
My intention is to make it so that a failed resolution (missing mapping) doesn't get added to the "resolved module set", so that it can be added in the future. I'll push a commit to that effect shortly.
Comment by @yoavweiss Oct 16, 2024 (See Github)
My intention is to make it so that a failed resolution (missing mapping) doesn't get added to the "resolved module set", so that it can be added in the future. I'll push a commit to that effect shortly.
https://github.com/whatwg/html/pull/10528/commits/dd0ff09eb0dadfa37715089ee38aa4423d23ce93 handled that logic. I'd appreciate a review to see that we're aligned on this point.
Comment by @martinthomson Oct 17, 2024 (See Github)
I don't know if I understand well enough to review. As far as I can see, you commit something if result
OR asURL
are non-null. That asURL
bit seems off, but I wasn't able to confirm. (Funny how GitHub stops working properly for like 110k of a file, even if the diff is modest in size, almost like it is a bad idea to have files that large).
Comment by @yoavweiss Oct 17, 2024 (See Github)
That
asURL
bit seems off, but I wasn't able to confirm
asURL
is null in case that the URL is not a URL-like specifier (essentially, if we fail to parse the specifier as a URL)
Comment by @jyasskin Oct 22, 2024 (See Github)
Thank you for the discussion. We're happy to see this proceed through the HTML review process.
Comment by @yoavweiss Oct 22, 2024 (See Github)
Thanks for reviewing!! :)
OpenedAug 9, 2024
こんにちは TAG-さん!
I'm requesting a TAG review of multple import maps (née "dynamic import maps").
Import maps currently have to load before any ES module and there can only be a single import map per document. That makes them fragile and potentially slow to use in real-life scenarios: Any module that loads before them breaks the entire app, and in apps with many modules the become a large blocking resource, as the entire map for all possible modules needs to load first.
This proposal is to enable multiple import maps per document, by merging them in a consistent and deterministic way.
Further details: