Git clone does not work in Windows Docker container in mounted host volume folder

[StefanScherer]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

Step 5/10 : RUN git --version --build-options
 ---> Running in 3d9c46cb4a76
git version 2.11.0.windows.1
sizeof-long: 4
machine: x86_64

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

Step 6/10 : RUN cmd.exe /c ver
 ---> Running in 2c465adb3865

Microsoft Windows [Version 10.0.14393]

• What options did you set as part of the installation? Or did you choose the
  defaults?

Step 7/10 : RUN type 'C:\Program Files\Git\etc\install-options.txt'
 ---> Running in ded3512e22b7
Path Option: CmdTools
SSH Option: OpenSSH
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Disabled
Enable Builtin Difftool: Disabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

Running Git for Windows in a Windows Container directly on a volume mount point.

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

PowerShell

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

Running inside a Windows container works fine:

PS C:\> docker run git-for-windows-issue git clone https://chromium.googlesource.com/external/gyp
Cloning into 'gyp'...

Running in a Windows container with a volume mounted from the Windows Docker host does not work:

PS C:\> docker run -v "$(pwd):C:\work" git-for-windows-issue git clone https://chromium.googlesource.com/external/gyp
Cloning into 'gyp'...
fatal: Could not switch to '/ContainerMappedDirectories/': No such file or directory

The Container directory C:\work is mounted from the Docker host. This can be shown with the dir command like this:

PS C:\> docker run -v "$(pwd):C:\work" git-for-windows-issue cmd /c dir ..
 Volume in drive C has no label.
 Volume Serial Number is 464C-9AB1

 Directory of C:\

11/22/2016  02:45 PM             1,894 License.txt
07/16/2016  05:18 AM    <DIR>          PerfLogs
12/24/2016  02:12 AM    <DIR>          Program Files
07/16/2016  05:18 AM    <DIR>          Program Files (x86)
12/24/2016  02:12 AM    <DIR>          Users
12/24/2016  02:12 AM    <DIR>          Windows
12/24/2016  02:06 AM    <SYMLINKD>     work [\\?\ContainerMappedDirectories\E5252A88-1539-401E-96FA-005C640978DB]
               1 File(s)          1,894 bytes
               6 Dir(s)  21,271,777,280 bytes free

It seems to me that Git is converting the UNC path into something and tries to switch to the "directory" ContainerMappedDirectories.

Running in a Windows container with a volume mounted from the Windows Docker host works if I use a sub directory below that mount point:

PS C:\> cat .\test-subdir.ps1
if (!(Test-Path subdir)) {
  mkdir subdir
}
cd subdir
git clone https://chromium.googlesource.com/external/gyp

PS C:\> docker run -v "$(pwd):C:\work" git-for-windows-issue powershell -file .\test-subdir.ps1


    Directory: C:\work


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       12/24/2016   2:25 AM                subdir
Cloning into 'gyp'...
Checking out files: 100% (1645/1645), done.

# back on the host:

PS C:\> dir .\subdir\


    Directory: C:\vagrant\resources\dockerfiles-windows\git-for-windows-issue\subdir


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----       12/24/2016   2:26 AM                gyp

• What did you expect to occur after running these commands?

Git clone should work directly in the mounted volume directory. I encountered that problem trying to build and test libuv in a container on a mapped volume. They have a command to clone into a sub directory git clone https://chromium.googlesource.com/external/gyp build/gyp but that also doesn't work if current dir in the container is the volume mount point.

• What actually happened instead?

fatal: Could not switch to '/ContainerMappedDirectories/': No such file or directory

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

No specific repo. But I have put this test scenario at https://github.com/StefanScherer/dockerfiles-windows/tree/master/git-for-windows-issue with the build and test scripts.

To build the Windows Docker image this Dockerfile is needed

FROM microsoft/windowsservercore

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

ENV chocolateyUseWindowsCompression false

RUN iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1')); \
    choco install -y git -params "/GitAndUnixToolsOnPath"

RUN git --version --build-options
RUN cmd.exe /c ver
RUN type 'C:\Program Files\Git\etc\install-options.txt'

# these install-options.txt files do not exist
# RUN type 'C:\Program Files (x86)\Git\etc\install-options.txt'
# RUN type ('{0}\AppData\Local\Programs\Git\etc\install-options.txt' -f $env:USERPROFILE)
# RUN cat /etc/install-options.txt

RUN mkdir C:\work
WORKDIR C:\\work

and then build the Docker image with

PS C:\> docker build -t git-for-windows-issue .

[dscho]

Thank you for this very thorough report.

I won't be near my laptop until next year, so I can only hope that other developers can reproduce the issue in the meantime.

One request from my side before that, though: we fixed a UNC path issue recently and I wonder whether it magically fixes the bug you reported, too. Would you mind testing again with https://github.com/git-for-windows/git/releases/tag/v2.11.1.windows-prerelease.1 ?

[StefanScherer]

Thanks. I've tried the v2.11.1.windows-prerelease.1, but there is the same error message.

No hurry, Docker on Windows is all new. Interesting enough I started to investigate a problem in Node.js which is very similar nodejs/node#8897, digging down to libuv and the build script there tries to clone a git repo and so I found this problem ;-)

[dscho]

Okay. Would you mind pinging me after Jan 2nd so this issue does not fall off my radar?

[vcarluer]

I got the same issue. It happens when I try to clone into a mounted volume or a subdirectory of it.

[asmagin]

I have the same issue. In my case, this is Jenkins that is trying to check-out code before a build. My JENKINS_HOME folder is mounted as a volume.

[StefanScherer]

There is a workaround for programs that want to retrieve the real path of a file or directory. Use a mapped drive letter of that volume and the program stops at the drive letter without stumbling at the volume reparse point.

See an example in my slides:

http://stefanscherer.github.io/windows-docker-workshop/#104

[dscho]

The culprit seems to be that DeviceIoControl() (which we use to parse the target of the symlink) returns a struct whose substitute name (which is supposedly the target) starts with a single backslash, not with the \\?\ prefix. No idea how to get that prefix (we could use the print name of course, but that is specifically intended for displaying, not for substituting...).

[StefanScherer]

The \\?\ only means that it is a long path name that may exceed the MAX_PATH. See https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx for details.

[dscho]

The \?\ only means that it is a long path name that may exceed the MAX_PATH.

It means that (we use that in core.longpaths support), but it means a lot more than that, too.

However, please note that in this case, it is a red herring. There is no such path... the output of dir shows this:

[...]
04/04/2017  10:29 PM    <SYMLINKD>     work [\\?\ContainerMappedDirectories\2AE72BC5-0AFB-4D2B-8A57-49F06FDF77C0]
[...]

However, the link target as per DeviceIoControl() reports \ContainerMappedDirectories\2AE72BC5-0AFB-4D2B-8A57-49F06FDF77C0 (note the missing \\? prefix). Worse: this is obviously not the path we want to use instead of C:\work.

[dscho]

And here is a related docker/docker ticket: moby/moby#31089

[StefanScherer]

Just found this comment https://go-review.googlesource.com/c/41834/ in a commit to fix it for Golang.

Currently windows Stat uses combination of Lstat and Readlink to
walk symlinks until it reaches file or directory. Windows Readlink
is implemented via Windows DeviceIoControl(FSCTL_GET_REPARSE_POINT, ...)
call, but that call does not work on network shares or inside of
Docker container (see issues #18555 ad #19922 for details).

But Raymond Chen suggests different approach:
https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/

• he suggests to use Windows I/O manager to dereferences the
  symbolic link.

This appears to work for all normal symlinks, but also for network
shares and inside of Docker container.