Skip to content

implement native for windows - #738

Closed
kentookura wants to merge 2 commits into
ocaml-multicore:mainfrom
kentookura:main
Closed

implement native for windows#738
kentookura wants to merge 2 commits into
ocaml-multicore:mainfrom
kentookura:main

Conversation

@kentookura

Copy link
Copy Markdown
Contributor

The implementation is copied from lib_eio_posix/fs.ml, simply replacing a forward slash.

I added the test case to the windows directory since the cross-platform test has some hard-coded forward slashes.

The output of the test reads

+<fs> -> Some .
+<fs:\\> -> Some \
+<fs:\\etc\\hosts> -> Some \etc\hosts
+<fs:.> -> Some .
+<fs:foo\\bar> -> Some .\foo\bar
+<cwd> -> Some .
+<cwd:..> -> Some .\..
+<native-sub> -> Some
+                  \??\C:\Users\asdf\eio\_build\default\lib_eio_windows\test\native-sub\
+<native-sub:foo.txt> -> Some
+                          \??\C:\Users\asdf\eio\_build\default\lib_eio_windows\test\native-sub\foo.txt
+<native-sub:.> -> Some
+                    \??\C:\Users\asdf\eio\_build\default\lib_eio_windows\test\native-sub\.
+<native-sub:..> -> Some
+                     \??\C:\Users\asdf\eio\_build\default\lib_eio_windows\test\native-sub\..
+<native-sub:\\etc\\passwd> -> Some \etc\passwd

I don't quite understand what \??\ is all about.

(* \\??\\ Is necessary with NtCreateFile. *)

@kentookura

Copy link
Copy Markdown
Contributor Author

Hacking.md states that the cross-platform tests "are run against whichever backend Eio_main.run selects, and therefore must get the same result for all backends", so I am a bit confused about how to correctly set up the new tests, as native was previously not available for windows, and the tests for native assume forward slashes.

@talex5

talex5 commented Jun 14, 2024

Copy link
Copy Markdown
Collaborator

Thanks for working on this!

Currently the tests directory isn't run on Windows (I think due to realworldocaml/mdx#295, though it's unclear what the problem is). native will indeed give different results on Windows, but we can use the new os_type feature added by @polytypic (realworldocaml/mdx#433) to skip the tests when they don't apply.

Windows supports forward and backward slashes in paths, and the idea is that an Eio.Path.t always uses forward slashes internally. So probably native needs to convert all of them to backward slashes for display.

I have no idea what the \\??\\ thing is about - @patricoferris wrote that code I think.

@kentookura

Copy link
Copy Markdown
Contributor Author

Alright, so commenting out the line int test/dune that disables the mdx tests on windows causes 100% CPU and memory usage for 10+ minutes with no end in sight.

@patricoferris

Copy link
Copy Markdown
Collaborator

I have no idea what the \??\ thing is about - @patricoferris wrote that code I think.

I think we can definitely improve on the Windows code and file path handling !

My understanding (helped plenty after speaking with @dra27) is these are NT Object Manager paths. Windows, to the normal user, has no real notion of "root" (/ on Unix-y systems). In that case, what exactly does Eio.Stdenv.fs env give to Window's users?

In order to use lower-level system functions like NtCreate... (to provide an openat-style interface) we have to use these NT paths.

avsm added a commit to avsm/eio that referenced this pull request Jul 28, 2026
Incorporates the approach from ocaml-multicore#738.

Co-authored-by: Kento Okura <kentookura@gmail.com>
avsm added a commit to avsm/eio that referenced this pull request Jul 28, 2026
Incorporates the approach from ocaml-multicore#738.

Co-authored-by: Kento Okura <kentookura@gmail.com>
avsm added a commit to avsm/eio that referenced this pull request Jul 28, 2026
Incorporates the approach from ocaml-multicore#738.

Co-authored-by: Kento Okura <kentookura@gmail.com>
avsm added a commit to avsm/eio that referenced this pull request Jul 28, 2026
Incorporates the approach from ocaml-multicore#738.

Co-authored-by: Kento Okura <kentookura@gmail.com>
avsm added a commit to avsm/eio that referenced this pull request Jul 28, 2026
Incorporates the approach from ocaml-multicore#738.

Co-authored-by: Kento Okura <kentookura@gmail.com>
Reviewed-by: Thomas Leonard <talex5@gmail.com>
avsm added a commit to avsm/eio that referenced this pull request Jul 29, 2026
Incorporates the approach from ocaml-multicore#738.

Co-authored-by: Kento Okura <kentookura@gmail.com>
Reviewed-by: Thomas Leonard <talex5@gmail.com>
talex5 pushed a commit to talex5/eio that referenced this pull request Jul 31, 2026
Incorporates the approach from ocaml-multicore#738.

This also makes a behavioural change I noticed while differentially
checking Go/Rust/Python. We prefer a \ when serialising Windows paths,
even though / is technically allowed. This seems less surprising

Co-authored-by: Kento Okura <kentookura@gmail.com>
talex5 pushed a commit to talex5/eio that referenced this pull request Jul 31, 2026
Incorporates the approach from ocaml-multicore#738.

This also makes a behavioural change I noticed while differentially
checking Go/Rust/Python. We prefer a \ when serialising Windows paths,
even though / is technically allowed. This seems less surprising

Co-authored-by: Kento Okura <kentookura@gmail.com>
talex5 pushed a commit to talex5/eio that referenced this pull request Aug 3, 2026
Incorporates the approach from ocaml-multicore#738.

This also makes a behavioural change I noticed while differentially
checking Go/Rust/Python. We prefer a \ when serialising Windows paths,
even though / is technically allowed. This seems less surprising

Co-authored-by: Kento Okura <kentookura@gmail.com>
talex5 pushed a commit to talex5/eio that referenced this pull request Aug 3, 2026
Incorporates the approach from ocaml-multicore#738.

This also makes a behavioural change I noticed while differentially
checking Go/Rust/Python. We prefer a \ when serialising Windows paths,
even though / is technically allowed. This seems less surprising

Co-authored-by: Kento Okura <kentookura@gmail.com>
talex5 pushed a commit to talex5/eio that referenced this pull request Aug 4, 2026
Incorporates the approach from ocaml-multicore#738.

This also makes a behavioural change I noticed while differentially
checking Go/Rust/Python. We prefer a \ when serialising Windows paths,
even though / is technically allowed. This seems less surprising

Co-authored-by: Kento Okura <kentookura@gmail.com>
talex5 pushed a commit to talex5/eio that referenced this pull request Aug 4, 2026
Incorporates the approach from ocaml-multicore#738.

This also makes a behavioural change I noticed while differentially
checking Go/Rust/Python. We prefer a \ when serialising Windows paths,
even though / is technically allowed. This seems less surprising

Co-authored-by: Kento Okura <kentookura@gmail.com>
talex5 pushed a commit to talex5/eio that referenced this pull request Aug 4, 2026
Incorporates the approach from ocaml-multicore#738.

This also makes a behavioural change I noticed while differentially
checking Go/Rust/Python. We prefer a \ when serialising Windows paths,
even though / is technically allowed. This seems less surprising

Co-authored-by: Kento Okura <kentookura@gmail.com>
@talex5

talex5 commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

This was included as part of #916 - thanks!

@talex5 talex5 closed this Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants