buildx bake: Specifying --file in another directory leads to failure

[sfgeorge]

Behaviour

tl;dr I can get docker buildx bake to succeed only when I cd to the subdirectory where docker-compose.yml lives.

Steps to reproduce this issue

Given the following project directory structure:

tests/docker-compose.yml
tests/dockerfiles/debian8/Dockerfile
tests/dockerfiles/debian9/Dockerfile
tests/dockerfiles/debian10/Dockerfile

And the following tests/docker-compose.yml file

version: "3"
services:
  debian8:
    build: ./dockerfiles/debian8
    cap_add: [ALL]
  debian9:
    build: ./dockerfiles/debian9
    cap_add: [ALL]
  debian10:
    build: ./dockerfiles/debian10
    cap_add: [ALL]

When a build definition file is specified in a subdirectory of the project such as above

Then resolution of Dockerfiles fails:

docker buildx bake -f  tests/docker-compose.yml

[+] Building 0.0s (0/0)
error: unable to prepare context: path "dockerfiles/debian9" not found

Yes, there is an obvious workaround – Simply first change the working directory to be that of where the build definition file exists.

cd tests && docker buildx bake

cd tests && docker buildx bake
[+] Building 1.4s (23/23) FINISHED
# (output omitted)

However, this limits the utility of this tool, especially in places like https://github.com/docker/bake-action, where that workaround is not possible because the option to change directories is not provided.

Expected behaviour

docker buildx bake -f ./tests/docker-compose.yml should succeed, given the context in the scenario above.

Configuration

• See this GitHub Actions Run: https://github.com/sfgeorge/rvm1-ansible/runs/5707843598?check_suite_focus=true

[crazy-max]

@sfgeorge I have made some tests: https://github.com/crazy-max/buildx-buildkit-tests/tree/main/buildx-1028

$ docker buildx bake -f tests/docker-compose.yml --print
{
  "group": {
    "default": {
      "targets": [
        "debian9",
        "debian8"
      ]
    }
  },
  "target": {
    "debian8": {
      "context": "dockerfiles/debian8",
      "dockerfile": "Dockerfile"
    },
    "debian9": {
      "context": "dockerfiles/debian9",
      "dockerfile": "Dockerfile"
    }
  }
}

$ docker buildx bake -f tests/docker-compose.yml
WARNING: No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
error: unable to prepare context: path "dockerfiles/debian9" not found

Using HCL:

$ docker buildx bake -f tests/docker-bake.hcl --print
{
  "group": {
    "default": {
      "targets": [
        "debian8",
        "debian9"
      ]
    }
  },
  "target": {
    "debian8": {
      "context": "dockerfiles/debian8",
      "dockerfile": "Dockerfile"
    },
    "debian9": {
      "context": "dockerfiles/debian8",
      "dockerfile": "Dockerfile"
    }
  }
}

$ docker buildx bake -f tests/docker-bake.hcl
WARNING: No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
error: unable to prepare context: path "dockerfiles/debian8" not found

Compose build:

$ docker compose -f tests/docker-compose.yml config
name: tests
services:
  debian8:
    build:
      context: /home/.../buildx-buildkit-tests/buildx-1028/tests/dockerfiles/debian8
      dockerfile: Dockerfile
    networks:
      default: null
  debian9:
    build:
      context: /home/.../buildx-buildkit-tests/buildx-1028/tests/dockerfiles/debian9
      dockerfile: Dockerfile
    networks:
      default: null
networks:
  default:
    name: tests_default

$ docker compose -f tests/docker-compose.yml build
Sending build context to Docker daemon     143B
Step 1/2 : FROM alpine
 ---> 76c8fb57b6fc
Step 2/2 : RUN echo debian8
 ---> Running in caae84c97f4a
debian8
Removing intermediate container caae84c97f4a
 ---> f63d9140e836
Successfully built f63d9140e836
Successfully tagged tests_debian8:latest
Sending build context to Docker daemon     143B
Step 1/2 : FROM alpine
 ---> 76c8fb57b6fc
Step 2/2 : RUN echo debian9
 ---> Running in 71b05a51a25b
debian9
Removing intermediate container 71b05a51a25b
 ---> 69365f840aca
Successfully built 69365f840aca
Successfully tagged tests_debian9:latest

It seems that docker compose defines the working directory where the compose file is located. Wonder if we should do the same with bake. WDYT @tonistiigi?

[sfgeorge]

Thanks so much for testing this out with me @crazy-max ! 🙌

[thaJeztah]

Yes, paths in compose-files are relative to the file. That said, things can get more complicated when multiple compose files are used from different locations, e.g. docker compose -f ./file1.yml -f ./subdir/file2/yml in that case, the files are merged, before evaluated, and (IIRC) evaluated based on the location of the first file.

I have some scratch note somewhere to propose having compose variables to have more control over this (e.g. to allow using {file-dir} or {project-dir} within the compose file to be clearer on intent, but need to give it some more thought before opening as a proposal in the compose-spec

[tonistiigi]

@tiborvass I remember you looked into what makes here. Do you remember where it was.

As @thaJeztah mentioned one of the cases that gets confusing is multi-file. As another example I think it is useful if you can do something like docker buildx bake -f ~/.buildx/lint.hcl and then run it in the context of any project.

I don't think I would be against having a predefined var that can be used for paths relative to the file itself.

[thaJeztah]

I had a proposal that I was writing; it's not fully finished, but I dusted it off a bit, and just posted it in compose-spec/compose-spec#248)

[paslandau]

I just ran into the same issue and use a workaround via --set *.context=. to fix it. Didn't see this yet anywhere so'm documenting it here.

Our project is set up like this

 .docker/
├──  docker-compose/
        ├── docker-compose.prod.yml
        └── docker-compose.yml
└── images/
        └──  foo/
                └── Dockerfile

docker-compose.yml

services:
  foo:
    build:
     # context is relative to the location of the file
     context: ../../
     dockerfile: ./.docker/images/foo/Dockerfile

and we invoke builds via compose build with

docker compose -f  .docker/docker-compose/docker-compose.yml  -f  .docker/docker-compose/docker-compose.prod.yml build 

When trying the same thing via buildx bake

docker buildx bake -f  .docker/docker-compose/docker-compose.yml  -f  .docker/docker-compose/docker-compose.prod.yml

I get the error

ERROR: could not find ..\..\.docker\images\foo: CreateFile ..\..\.docker\images\foo: The system cannot find the path specified.

My current workaround is using --set *.context=. to override the context to . (see https://docs.docker.com/engine/reference/commandline/buildx_bake/#set )

docker buildx bake -f  .docker/docker-compose/docker-compose.yml  -f  .docker/docker-compose/docker-compose.prod.yml --set *.context=.

FYI: This "works" for now, because all docker compose config files use the same context ../../ ==> this might not always be the case and require an adoption of the compose config files (and Dockerfiles, because COPY commands would need to change).

[olee]

There are actually more issues with bake and relative paths - in this case with additional_contexts:
I have a folder structure like this:

├── api/
│   ├── Dockerfile
│   └── docker-compose.yaml
├── frontend/
│   ├── Dockerfile
│   └── docker-compose.yaml
├── shared/
│   └── <some-files>
└── docker-compose.yaml

Where the root docker-compose.yaml looks like this:

version: "3.8"
services:
  api:
    extends:
      file: ./api/docker-compose.yaml
      service: api
  frontend:
    extends:
      file: ./frontend/docker-compose.yaml
      service: frontend

And a package's docker-compose.yaml looks like this:

version: "3.8"
services:
  api:
    image: my-api-image
    build:
      context: .
      additional_contexts:
        # Provide access to shared code
        shared: ../shared

This works extremly well with docker compose, but bake will fail because it tries to lookup the shared directory not relative to the dockerfile it was specified in, but to the root directory instead.
Right now I am fixing this by providing an additional bake file to merge with the default docker-compose.yaml with this type of adjustment:

services:
  api:
    build:
      additional_contexts:
        shared: ./shared
  frontend:
    build:
      additional_contexts:
        shared: ./shared

[rizary]

I have the same error on windows, where I put:

    build:
      dockerfile: ../../images/Dockerfile.web-dev
      context: ../../../web

then I get the below error:

failed to evaluate path "D:\Engineer\web3\rizilab\services/D:\Engineer\web3\rizilab\services\images\Dockerfile.web-dev": CreateFile D:\Engineer\web3\rizilab\services\D:: The filename, directory name, or volume label syntax is incorrect.

when i remove COMPOSE_BAKE=true, everything works perfectly

[r-via]

Same problem, here on windows