#805: Moving local files with the File System Access API
Discussions
Discussed
Feb 1, 2023 (See Github)
bump
Discussed
Mar 1, 2023 (See Github)
Dan: reprises the convo in breakout C
Lea: this is important - parity with native platforms -
Yves: we didn't say rule it out - but trying to get others (even apple and mozilla) to figure out a way to make acceptable for the user... how to design it properly?
Lea: maybe a different level of permission dialog... shouldn't be on the same level of notifications
Dan: or geolocation...
Lea: although i'm not sure that "show file picker" needs to be behind that...
Peter: i think you have to have a picker... file or directory...
Lea: then there is very clear user action...
Peter: moving talks about origin file system ...
Lea: this model of having APIs live on the general web and trying to get web apps to compete with native apps... clearly not working. Maybe this needs to be solved on the architectural level. Explicit user action "I want to "
Dan: i think it's explicit. .. but maybe worth revisiting.
Peter: i don't ever want a web app to have unfettered access to my entire filesystem... even native apps are getting locked out by OSs in some cases... this is somewhat scoped... there are things like file pickers... once the user has picked a file or directory then the app can inetract with that directory. If the move api can only access directories that the user has previously allowed then that's OK.
Hadley: user need?
Dan: one of the documented user needs is moving or renaming a big video file...
Hadley: i can see about renaming... don't know about moving files though.
Yves: during the call we discussed if renaming could restrict to same directory only...
Lea: i think web apps should be able to do things that other apps do.. right no the arch doesn't allow for this.
Peter: I agree - and - I think native apps have too much freedom. The web is a different conetxt. There is the guarantee of the UA providing a sandbox.
debate on the nature of the web and the nature of nature
Hadley: I agree with Peter's point, but also.. example of google doc on a native word processor, saving a document to local disk is trivial... on a web based word processor it's not, since you have to download the file to downloads folder and then move to the right directory in the OS (outside of the web).. as a user that's frustrating...
Lea: input type=file... hack... it's bad for accessibility...
Peter: there is a show open picker API .. what does that not give you that that hack does?
Discussed
Mar 1, 2023 (See Github)
Yves: negative standards position from Mozilla on this one.
Dan: explainer and spec points to PRs which is making things difficult.
Yves: this is local file system...
Yves: Webkit position is negative as well...
discussion on whether this is OPFS or not
Sangwhan: the use cases are in explainer
Dan: Security Consideration - how does a site gain explicit write access to a file? I'm very concerned about security mitigations...
Yves: it's not only ststem files... directories in home directory that set environemntal variables etc... Lots of data writable in user directory. Once you know you're in a safe environment then move() is a no-brainer but ...
Sangwhan: There is this
Dan: "might want restrict" is loosey-goosey
Yves: "default download" is difficult - you may have sensitive info such as statements from your bank...
Sangwhan: move can be a destructive operation...
Dan: we previously reviewed https://github.com/w3ctag/design-reviews/issues/390 and this became native filesystem?
Sangwhan: yes. Positive. There were multiple iterations... we arised issues. they addressed our feedback. Not everyone in TAG was happy. But we discussed it as a group and decided it was probably OK.
Dan: concerns about mitigations in file system access...
Yves: having user agent showing you files in your directory... but doing the same from javascript is very different because you're already leaking some information. Unless there are good security requirements -enforceable security aspects in the spec. Move() is not really the issue...
Hi @a-sully - thanks for this. We're reviewing today and noting that this is built on top of File System Access which we previously [reviewed positively](https://github.com/w3ctag/design-reviews/issues/390). Can we set up a session where y'all join us to discuss the security issues, abuse cases, and potential mitigations?
Sangwhan: is there a way to have secure file access besides OPFS? It seems to me the answer is no. Philosophical question ...
... for this API to enable malware then the malware would have to access on the disk. Even OPFS if you have a file handle you can push malware.
Yves: recommendation of multi-stakeholder support and that includes security aspects because that's where apple and mozilla have issues as well... If we fix the security aspect - won't be easy but it's doable - then it will provide lots of benefit.
Sangwahn: Expresses concerns about this pushback... I don't see how the issues can be mitigated...
Discussed
Mar 1, 2023 (See Github)
Dan: I will follow up.
Discussed
Mar 1, 2023 (See Github)
Dan: I think Sangwhan was going to talk to the developers about strengthening the language around threats and abuse cases so let's wait until he updates us on that.
Comment by @torgo Mar 14, 2023 (See Github)
Comment by @torgo Mar 14, 2023 (See Github)
Comment by @torgo Mar 14, 2023 (See Github)
Hi @a-sully – are there versions of the spec and explainer markdowns that can be directly linked to, rather than un-merged PRs? That would greatly help the review.
Comment by @torgo Mar 14, 2023 (See Github)
Hi @a-sully - thanks for this. We're reviewing today and noting that this is built on top of File System Access which we previously reviewed positively. Can we set up a session where y'all join us to discuss the security issues, multi-stakeholder interest, abuse cases, and potential mitigations?
Comment by @plinss Mar 15, 2023 (See Github)
I'm not sure if this is already covered, but there needs to be a limitation on how often this method can be called. Likely restricted to a single call per user activation.
Otherwise, a malicious site can use the fact that existing files can't be overwritten to probe for the existence of other files that the user has not granted access to.
Comment by @a-sully Mar 16, 2023 (See Github)
Hi @a-sully – are there versions of the spec and explainer markdowns that can be directly linked to, rather than un-merged PRs? That would greatly help the review.
No, not at the moment. There had been some earlier discussions as to where these types of explainers should go but that thread was never resolved... I can attempt to merge them into whatwg/fs and cite this as justification :)
Hi @a-sully - thanks for this. We're reviewing today and noting that this is built on top of File System Access which we previously reviewed positively. Can we set up a session where y'all join us to discuss the security issues, multi-stakeholder interest, abuse cases, and potential mitigations?
Absolutely! (and sorry, I realized I had accidentally been filtering emails related to this repo. I should be more responsive going forward)
I'm not sure if this is already covered, but there needs to be a limitation on how often this method can be called. Likely restricted to a single call per user activation.
Otherwise, a malicious site can use the fact that existing files can't be overwritten to probe for the existence of other files that the user has not granted access to.
Correct! This is discussed in the explainer; both on requiring user activation if you don't have permission and overwriting existing files.
We expect this to be sufficient, but can add usage limitations later on if needed
Discussed
Apr 1, 2023 (See Github)
we discussed some issues the current state of the review and process - e.g. lack of an expaliner and any prviacy & security considerations section
Comment by @a-sully Apr 4, 2023 (See Github)
Hi @a-sully – are there versions of the spec and explainer markdowns that can be directly linked to, rather than un-merged PRs? That would greatly help the review.
No, not at the moment. There had been some earlier discussions as to where these types of explainers should go but that thread was never resolved... I can attempt to merge them into whatwg/fs and cite this as justification :)
Following up on this - see https://github.com/whatwg/fs/pull/108
In that PR I mention "the anti-pattern of putting the useful text only in the explainers and not in the spec text itself". Unfortunately this feature is currently in a worse state than even that, since the spec PR for this feature (https://github.com/whatwg/fs/pull/10) has yet to land at all. For months, specifying move()
(among many other things) was blocked on answering the fundamental question posed in https://github.com/whatwg/fs/issues/59 of what exactly is a FileSystemHandle
Fortunately, this was just recently resolved in https://github.com/whatwg/fs/pull/96. There's now only one structural blocker to getting the move()
method specified, which I hope will be merged soon: https://github.com/whatwg/fs/pull/95
In the meantime, I'll be updating https://github.com/whatwg/fs/pull/10 to include more information about the restrictions discussed in the hopefully-soon-to-be-merged explainers/MovingNonOpfsFiles.md
Apologies for the chaos! The move()
method has a complicated history (it's been shipped in Chromium browsers for only files in the OPFS for about a year now) that I'm still working to make right. Please don't hesitate to reach out if it would be useful for me to join a session with y'all
Comment by @a-sully Apr 20, 2023 (See Github)
FYI: the explainer for this feature has a new home https://github.com/whatwg/fs/blob/main/proposals/MovingNonOpfsFiles.md
Discussed
May 1, 2023 (See Github)
Dan: back and forth on security concerns. Maybe worth putting this into the issue. In the explainer it says user agents are recommended to perform checks on files moved. There's no spec, only an explainer. S&P considerations is in another spec somewhere else.
<blockquote> Hi - One thing that is still blocking this from our perspective is that the Spec is not self-contained so it's difficult to understand what we're reviewing - especially the privacy & security considerations. In the explainer it says "User agents are recommended to perform security checks on files moved within the local file system" but that isn't in the linked PR. And one issue we're concerned about is the strength of this "recommendation" and whether it's appropriate to the power of this API - especially when it comes to security. If there's a self-contained spec, can you please amend the review to point to that? </blockquote>Dan: leaves comment
Comment by @torgo May 16, 2023 (See Github)
Hi - Thanks for the update on the explainer. One thing that is still blocking this from our perspective is that the Spec is not self-contained so it's difficult to understand what we're reviewing - especially the privacy & security considerations. In the explainer it says "User agents are recommended to perform security checks on files moved within the local file system" but that isn't in the linked PR. And one issue we're concerned about is the strength of this "recommendation" and whether it's appropriate to the power of this API - especially when it comes to security. If there's a self-contained spec, can you please amend the review to point to that?
Comment by @a-sully Jun 30, 2023 (See Github)
Hi @torgo,
The move()
spec PR has been blocked on several infrastructural changes and at this point is a bit out of date. I expect https://github.com/whatwg/fs/pull/131 is the last remaining piece we need to finally be able to specify it properly, at which point I'll go back and update the spec PR
Comment by @a-sully Jun 30, 2023 (See Github)
In the explainer it says "User agents are recommended to perform security checks on files moved within the local file system" but that isn't in the linked PR. And one issue we're concerned about is the strength of this "recommendation" and whether it's appropriate to the power of this API - especially when it comes to security.
Regarding this point specifically - we recent added to the spec the ability to distinguish between files on the local file system vs. files in a Bucket File System (f.k.a. OPFS). This now allows us to specify within the move()
algorithm something like:
- If [=this=] [=FileSystemHandle/is in a bucket file system=], run [=implementation-defined=] malware checks
Discussed
Jul 1, 2023 (See Github)
Dan: asking for clarity about distinguishing between local file system and Bucket file system. Does it add additional security guarantees for the local filesystem?
Sangwhan: what do normative security guarantees look like in a spec? Eg. if you say the UA must detect malware that's a wide net...
Dan: the user agent must not allow files to be moved into protected directories
Sangwhan: that's defined
Dan: it was not normative
Sangwhan: you'd have to ensure that the spec is always up to date to the OS specs. That becomes a bigger spec than the spec itself..
Dan: I don't think the spec needs to normatively define every location which is a protected directory. How does this make it clear to implementers that they must implement protections against this being used to manipulated files in protected areas? Arguably things like the downloads directory is a protected area, as Peter was mentioning. Can we ask them to strengthen the language? Based on this change they've made, which is great, that they can differentiate - what does that enable them to do to allow for protecting users? I can just ask that.
Sangwhan: fine to ask. It's difficult to define normatively.
Discussed
Jul 1, 2023 (See Github)
Dan: response from them..
Yves: I think other browsers won't support because they won't want to give access to filesystem outside of bucket file system in general...
Dan: My suggestion is to close this at the plenary with "satisfied with concerns" - specifically the concern about multi-browser support.
sets to proposed-closed
Discussed
Jul 1, 2023 (See Github)
mv *.zig # for great justice chown us:us *.base
Dan: Still pending...
bumped 2 weeks
Comment by @torgo Jul 11, 2023 (See Github)
Hi @a-sully - can you clarify what do you mean by "distinguish"? Do you mean that the spec itself would mandate a higher security level for anything to do with local file system files vs. "bucket file system" files? Is there additional normative spec text that could be therefore added to protect users from bad outcomes (i.e. necessary security guarantees)?
Comment by @a-sully Jul 12, 2023 (See Github)
can you clarify what do you mean by "distinguish"
We now have the ability to say "this FileSystemHandle
corresponds to a file in a bucket file system" (as opposed to the local file system). See https://fs.spec.whatwg.org/#filesystemhandle-is-in-a-bucket-file-system.
I expected the move()
method to be specified similarly to the existing specification for running malware checks when closing a FileSystemWritableFileStream
:
Run implementation-defined malware scans and safe browsing checks. If these checks fail, reject closeResult with an "AbortError" DOMException and abort these steps.
"Implementation defined" is broad enough that each implementation could choose to skip malware checks for files in the bucket file system - which I expect is the case currently across all browsers. We could be explicit that these checks must not run for files in the bucket file system (as I sketched out above). That would remove some theoretical flexibility on the part of the user agent, but in practice I don't expect that to be an issue (cc @jesup @szewai @annevk)
Meanwhile, there's the question @torgo posed about running security checks for local files ("necessary security guarantees"). From my perspective (please correct me if I'm misunderstanding), there are two pieces to this:
- Can we specify these security checks in more detail, say like downloads? (i.e. can we do better than just "implementation-defined"?)
- Should we specify that security checks must be run when saving or moving files on the local file system?
I don't think we want (2). See the non-normative language here: https://wicg.github.io/file-system-access/#security-malware
user agents are encouraged to verify the contents of files modified by this API via malware scans and safe browsing checks, unless some kind of external strong trust relation already exists
The user agent should have the flexibility to skip security checks in some scenarios (e.g. the user agent can determine what qualifies as a "strong trust relation", how it's established, etc)
Regarding (1), I'm open to discussing adding more detail (and since this also affects the existing specification of FileSystemWritableFileStream
, we may want to move this discussion to a new issue on the FS spec). But without (2) we can't guarantee any security behaviors
Comment by @a-sully Jul 17, 2023 (See Github)
I've just revamped https://github.com/whatwg/fs/pull/10/ to make use of the various infrastructural changes I mentioned above and to include the restrictions discussed in the explainer for non-BucketFileSystem moves. Feel free to leave comments here or on that PR
A few things to note:
- The steps for requesting access to the destination are still subject to change depending on what we decide in https://github.com/whatwg/fs/pull/131
- No browsers yet support directory moves outside of the BucketFileSystem, so there are likely more restrictions we'll want to add that - may require some acrobatics with the https://github.com/wicg/file-system-access spec. For example, we should not allow well-known directories to be moved
Comment by @torgo Jul 18, 2023 (See Github)
Hi @a-sully thanks for that and thanks for listening to our comments and concerns. Our main concern continues to be security-related issues related allowing access to the local filesystem. In the case where different UAs have made different security choices about this capability, does the API allow for graceful fallback when the browser in question doesn't support file moves in the local file system? Is that detectable? Have you thought about how this would behave differently with isolated web apps #842 ?
Discussed
Aug 1, 2023 (See Github)
Sangwhan to reach out
Discussed
Aug 1, 2023 (See Github)
Dan: Some new comments from @a-sully. They have addressed many of our concerns - in particular with regard to security considerations. There is still a multi-stakeholder issue because webkit and mozilla both oppose. The issue not to do with the move capability itself but the whole concept of access to local file system.
Dan: My proposed closing comment follows:
<blockquote> Hi folks - thanks for your responses and thanks for addressing our concerns. We're happy with the proposal as it stands, however I think the main concern we have now relates to multi-stakeholder support for the file system access API itself. We encourage you to work with other stakeholders / implementers to achieve a consensus-based approach to this capability. </blockquote>Dan: Suggesting satisfied with concerns result.
we agreed to close
Comment by @a-sully Aug 22, 2023 (See Github)
does the API allow for graceful fallback when the browser in question doesn't support file moves in the local file system
On browsers for which only the Bucket File System is supported, well, there is currently no way to get a FileSystemHandle
corresponding to the local file system at all. But if a browser wanted to support the local file system in a limited capacity - say, by allowing read-only access to files which are dragged-and-dropped or selected via a file picker - then the requirement of "readwrite" permission to (at least) the source file would result in the move()
method reliably throwing a NotAllowedError
exception
Is that detectable?
Were other browsers to implement (even partial) support for the local file system, that could include upstreaming the FileSystemHandle.queryPermission()
method (and possibly also the FileSystemHandle.requestPermission()
method) from the WICG spec to the whatwg spec. The website could then check queryPermission({ mode : "readwrite" })
before bothering to try to move()
the file
Have you thought about how this would behave differently with isolated web apps https://github.com/w3ctag/design-reviews/issues/842 ?
The permission requirements mentioned above will also apply to IWAs. If the user agent wanted to somehow privilege IWAs - say, by allowing only IWAs to call move()
- that would be facilitated by granting the appropriate File System permissions to the IWA. The means of handing out File System permissions for the local file system - to IWAs or otherwise - will presumably remain implementation-defined
Comment by @torgo Aug 29, 2023 (See Github)
Hi folks - thanks for your responses and thanks for addressing our concerns. We're happy with the proposal as it stands, however I think the main concern we have now relates to multi-stakeholder support for the file system access API itself. We encourage you to work with other stakeholders / implementers to achieve a consensus-based approach to this capability.
OpenedJan 17, 2023
Wotcher TAG!
I'm requesting a TAG review of
FileSystemFileHandle.move()
for local files.When launching SyncAccessHandles, we launched
FileSystemFileHandle.move()
for files within the Origin Private File System (OPFS). Moving of files outside of the OPFS and moving directories at all are not yet supported.We're proposing to allow the
FileSystemFileHandle.move()
method to move files that do not live in the Origin Private File System, i.e. user-visible files on the device.Further details:
We'd prefer the TAG provide feedback as:
🐛 open issues in our GitHub repo for each point of feedback