Relative paths in compose file not resolved correctly? #197

Activity

1. FernandoMiguel commented on Nov 29, 2019

   

   FernandoMiguel

   on Nov 29, 2019

   Contributor

   More actions

   that is correct and expected.
   the context is different.
   from BuildX perspective the root is the context, not the service.

   you can easily add an override yml

   😕React with 😕1Reacted by Teoh Han Hui
2. teohhanhui commented on Nov 29, 2019

   

   teohhanhui

   on Nov 29, 2019

   Author

   More actions

   It contradicts the Compose file documentation: https://docs.docker.com/compose/compose-file/#context

   When the value supplied is a relative path, it is interpreted as relative to the location of the Compose file.

   You risk breaking compatibility with many Compose files...

   👍React with 👍2Reacted by andoks and sephv
3. FernandoMiguel commented on Nov 29, 2019

   

   FernandoMiguel

   on Nov 29, 2019

   Contributor

   More actions

   there is no enforced compatibility with compose... only that BuildX is able to read them
   the same way it parses HCL and JSON.

   as i said, you can just as easily keep using your existing compose YAML, just add an override.
   I've done this in the past.
   super easy

   😕React with 😕1Reacted by Teoh Han Hui
4. tonistiigi commented on Dec 3, 2019

   

   tonistiigi

   on Dec 3, 2019

   Member

   More actions

   @tiborvass I remember we discussed how all of these paths should be resolved a while back. Do you have notes from that discussion?
5. tiborvass commented on Dec 4, 2019

   

   tiborvass

   on Dec 4, 2019

   Last edited by tiborvass

   Collaborator

   More actions

   There are valid arguments to the current path logic that bake uses, but I agree with @teohhanhui it is preferable to align ourselves with established docker-compose user expectations, so let's fix this.

   For keeping a written record, below is a table with all possible cases and proposed UX summing up conversations with @tonistiigi. Basically, we would detect a ./ prefix and interpret it as relative to working directory.

   With context path resolution:

   • buildx bake -f filename

   filename	context	resulting context path
   foo/docker-compose.yml	bar/	./foo/bar
   foo/docker-compose.yml	./bar/	./bar
   foo/docker-compose.yml	git://remote/repo	/tmp/git/repo
   http://remote/docker-compose.yml	bar/	./bar
   http://remote/docker-compose.yml	./bar/	./bar
   http://remote/docker-compose.yml	git://remote/repo	/tmp/git/repo

   (filename cannot be git:// because a file, not a folder has to be specified)

   Same with Dockerfile path resolution:

   • buildx bake mytarget
     • bakefile: {"mytarget": {"context" : "git://remote/repo", "dockerfile": "foo.Dockerfile"}}
       • Resulting dockerfile path: /tmp/git/repo/foo.Dockerfile
     • bakefile: {"mytarget": {"context" : "git://remote/repo", "dockerfile": "./foo.Dockerfile"}}
       • Resulting dockerfile path: ./foo.Dockerfile
   • buildx bake git://remote/bakery mytarget (would require detecting protocol prefix in target)
     • bakefile: {"mytarget": {"context" : "git://remote/repo", "dockerfile": "foo.Dockerfile"}}
       • Resulting dockerfile path: /tmp/git/repo/foo.Dockerfile
     • bakefile: {"mytarget": {"context" : "git://remote/repo", "dockerfile": "./foo.Dockerfile"}}
       • Resulting dockerfile path: /tmp/git/bakery/foo.Dockerfile

   /cc @ulyssessouza for harmonizing this suggested solution with docker-compose.
6. teohhanhui commented on Dec 4, 2019

   

   teohhanhui

   on Dec 4, 2019

   Last edited by teohhanhui

   Author

   More actions

   @tiborvass The different behaviour between bar/ and ./bar/ will really bite the users.

   . means the current directory, so if we follow:

   When the value supplied is a relative path, it is interpreted as relative to the location of the Compose file.

   they should not be different.

   If the intention is to allow anchoring to the current working directory, perhaps it should be something explicit like ${PWD}, instead of introducing subtle differences that will lead to bad DX.

   👍React with 👍5Reacted by andoks, John Munson, Ryan Wheale, Jesse Hallett and mubashirusman
7. andoks commented on Nov 5, 2021

   

   andoks

   on Nov 5, 2021

   Last edited by andoks

   More actions

   EDIT: sorry for first deleting my comment, then recreating it. But I started thinking this issue was specifically for the "context" property, but I then changed my mind. My issue still relates to the way relative paths and paths in general are specified/interpreted in docker-compose.

   I think I might have hit something that relates to this issue today when trying to mount files as volumes into a container using a docker-compose override file located in a subpath.

   This subpath also contains all the files to be mounted into the various containers. In my instance compose complains about ERROR: Named volume "cert.pem:/etc/cert.pem:rw" is used in service "myservice" but no declaration was found in the volumes section. when using the suggested form from @tiborvass 's comment of starting paths that are to be considered relative to the compose-file with no prefix.

   My expectation was that paths starting with ./ were to be considered relative to the docker-compose file, while the ${PWD} prefix specifies relative to current working directory.

   My versions of docker:

   Client:
    Version:           20.10.2
    API version:       1.41
    Go version:        go1.13.8
    Git commit:        20.10.2-0ubuntu1~20.10.1
    Built:             Tue Mar 23 23:21:32 2021
    OS/Arch:           linux/amd64
    Context:           default
    Experimental:      true
   
   Server:
    Engine:
     Version:          20.10.2
     API version:      1.41 (minimum version 1.12)
     Go version:       go1.13.8
     Git commit:       20.10.2-0ubuntu1~20.10.1
     Built:            Tue Mar 16 19:16:09 2021
     OS/Arch:          linux/amd64
     Experimental:     false
    containerd:
     Version:          1.5.2-0ubuntu1~20.10.2
     GitCommit:        
    runc:
     Version:          1.0.0~rc95-0ubuntu1~20.10.1
     GitCommit:        
    docker-init:
     Version:          0.19.0
     GitCommit:        
   
8. nirradi commented on Apr 14, 2022

   

   nirradi

   on Apr 14, 2022

   Last edited by nirradi

   More actions

   When using more than one docker-compose file, then there is inconsistent behavior in this aspect. The first docker-compose yml determines the working directory for path resolution, so files that work on their own suddenly stop working.

   example
   path1/file1.yml:

   version: "3"
   services: 
      servicea: 
         env_file:
           - dev.env
        
   
   

   path2/file2.yml

   version: "3"
     
   
   

   then this works

   docker-compose -f path1/file1.yml up 
   docker-compose -f path1/file1.yml -f path2/file2.yml up 
   

   and this doesn't

   docker-compose -f path2/file2.yml -f path1/file1.yml up 
   

   even when the working directory is path1

   👍React with 👍1Reacted by Jacob Thomason
9. crazy-max

   added

   area/bake

   area/bake/compose

   on Sep 10, 2022
10. tonistiigi

    closed this as completedin #3935on Jul 22, 2026