Skip to content

[BUG] docker compose up --build does not restart containers if image ID changes, making docker compose images fail #14014

Closed
#14028
Closed
[BUG] docker compose up --build does not restart containers if image ID changes, making docker compose images fail#14014
#14028
Assignees
ndeloof
Labels
kind/bug

Description

@felixfontein
felixfontein
opened on Aug 7, 2026

Description

In the CI for the Ansible collection community.docker, I noticed a problem with the latest Compose release in certain environments. I cannot reproduce it on my machine (Arch Linux) or with GHA's ubuntu:latest Docker daemon, but reproducible on RHEL 9.8 and 10.2 VMs.

It can be reproduced with the following docker-compose.yml:

services:
  foo:
    build: ./build
    image: foo
    pull_policy: never
    stop_grace_period: 1s

and the following build/Dockerfile:

FROM quay.io/ansible/docker-test-containers:alpine3.8
ENTRYPOINT ["/bin/sh", "-c", "sleep 10m"]

Steps To Reproduce

  1. Set up docker-compose.yml and build/Dockerfile as above.
  2. Run docker compose up --detach --build
  3. Run docker compose images: this works
  4. Run docker compose up --detach --build again
  5. Run docker compose images: this now fails with Error response from daemon: No such image: sha256:40bb5bcde0724b0fee1b54ab60b9685c9c04c9fc0fe70228d61653f8a43ac921 (ID will vary)

Compose Version

Docker Compose version v5.4.0

Docker Environment

Client: Docker Engine - Community
 Version:    29.7.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.36.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v5.4.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 1
  Running: 1
  Paused: 0
  Stopped: 0
 Images: 1
 Server Version: 29.7.2
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: aad11006b869517fcd3009450b6f82da282e1a9b
 runc version: v1.4.3-0-gbb14dabe
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.12.0-211.40.1.el10_2.x86_64
 Operating System: Red Hat Enterprise Linux 10.2 (Coughlan)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 3.47GiB
 Name: ip-192-168-3-57.ec2.internal
 ID: 03bc9448-54df-4252-b23a-1494eb3ff3f4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false
 Firewall Backend: iptables
  EnableUserlandProxy: true
  UserlandProxyPath: /usr/bin/docker-proxy

Anything else?

When running docker image list before and after the second docker compose up --detach --build run, I can see that the image ID changes. Despite that, the second docker compose up --detach --build does not restart the container. When running docker compose --ansi never --progress json up --detach --build to see what exactly is happening, I get something like:

{"id":"Image foo","status":"Working","text":"Building"}
{"vertexes":[{"digest":"sha256:4b58559d8a6864678e56d2ebb00a1cd35a60f47dc25d239de21796391ffab4ab","name":"[internal] load local bake definitions","started":"2026-08-07T08:44:46.375968748Z"}]}
{"statuses":[{"id":"reading from stdin","vertex":"sha256:4b58559d8a6864678e56d2ebb00a1cd35a60f47dc25d239de21796391ffab4ab","current":526,"timestamp":"0001-01-01T00:00:00Z","started":"2026-08-07T08:44:46.376063309Z"}]}
{"statuses":[{"id":"reading from stdin","vertex":"sha256:4b58559d8a6864678e56d2ebb00a1cd35a60f47dc25d239de21796391ffab4ab","current":526,"timestamp":"0001-01-01T00:00:00Z","started":"2026-08-07T08:44:46.376063309Z"}]}
{"statuses":[{"id":"reading from stdin","vertex":"sha256:4b58559d8a6864678e56d2ebb00a1cd35a60f47dc25d239de21796391ffab4ab","current":526,"timestamp":"0001-01-01T00:00:00Z","started":"2026-08-07T08:44:46.376063309Z","completed":"2026-08-07T08:44:46.37640108Z"}]}
{"vertexes":[{"digest":"sha256:4b58559d8a6864678e56d2ebb00a1cd35a60f47dc25d239de21796391ffab4ab","name":"[internal] load local bake definitions","started":"2026-08-07T08:44:46.375968748Z","completed":"2026-08-07T08:44:46.376402123Z"}]}
{"vertexes":[{"digest":"sha256:954f26cb6dd91bec763d3b09cd0952be8079cb23d68cef1738481228d8dfe9a0","name":"[internal] load build definition from Dockerfile"},{"digest":"sha256:954f26cb6dd91bec763d3b09cd0952be8079cb23d68cef1738481228d8dfe9a0","name":"[internal] load build definition from Dockerfile","started":"2026-08-07T08:44:46.40710106Z"}]}
{"vertexes":[{"digest":"sha256:954f26cb6dd91bec763d3b09cd0952be8079cb23d68cef1738481228d8dfe9a0","name":"[internal] load build definition from Dockerfile","started":"2026-08-07T08:44:46.40710106Z","completed":"2026-08-07T08:44:46.407123437Z"}]}
{"vertexes":[{"digest":"sha256:954f26cb6dd91bec763d3b09cd0952be8079cb23d68cef1738481228d8dfe9a0","name":"[internal] load build definition from Dockerfile","started":"2026-08-07T08:44:46.407188272Z"}]}
{"statuses":[{"id":"transferring dockerfile:","vertex":"sha256:954f26cb6dd91bec763d3b09cd0952be8079cb23d68cef1738481228d8dfe9a0","name":"transferring","current":0,"timestamp":"2026-08-07T08:44:46.408169451Z","started":"2026-08-07T08:44:46.408168733Z"}]}
{"statuses":[{"id":"transferring dockerfile:","vertex":"sha256:954f26cb6dd91bec763d3b09cd0952be8079cb23d68cef1738481228d8dfe9a0","name":"transferring","current":89,"timestamp":"2026-08-07T08:44:46.408815317Z","started":"2026-08-07T08:44:46.408168733Z"}]}
{"statuses":[{"id":"transferring dockerfile:","vertex":"sha256:954f26cb6dd91bec763d3b09cd0952be8079cb23d68cef1738481228d8dfe9a0","name":"transferring","current":193,"timestamp":"2026-08-07T08:44:46.409516781Z","started":"2026-08-07T08:44:46.408168733Z","completed":"2026-08-07T08:44:46.409516267Z"}]}
{"vertexes":[{"digest":"sha256:954f26cb6dd91bec763d3b09cd0952be8079cb23d68cef1738481228d8dfe9a0","name":"[internal] load build definition from Dockerfile","started":"2026-08-07T08:44:46.407188272Z","completed":"2026-08-07T08:44:46.412485734Z"}]}
{"vertexes":[{"digest":"sha256:da2b2b574ec5b30f5c1f85118b7dd7425faffe11eb54e347fbb459cfb256fb04","name":"[internal] load metadata for quay.io/ansible/docker-test-containers:alpine3.8","started":"2026-08-07T08:44:46.414410784Z"}]}
{"vertexes":[{"digest":"sha256:da2b2b574ec5b30f5c1f85118b7dd7425faffe11eb54e347fbb459cfb256fb04","name":"[internal] load metadata for quay.io/ansible/docker-test-containers:alpine3.8","started":"2026-08-07T08:44:46.414410784Z","completed":"2026-08-07T08:44:46.639776176Z"}]}
{"vertexes":[{"digest":"sha256:9972a57f977c23ca76125b42c1c405dee759e25b0492da93b67b0c4d4b55f249","name":"[internal] load .dockerignore"}]}
{"vertexes":[{"digest":"sha256:9972a57f977c23ca76125b42c1c405dee759e25b0492da93b67b0c4d4b55f249","name":"[internal] load .dockerignore","started":"2026-08-07T08:44:46.640346122Z","completed":"2026-08-07T08:44:46.640360508Z"},{"digest":"sha256:9972a57f977c23ca76125b42c1c405dee759e25b0492da93b67b0c4d4b55f249","name":"[internal] load .dockerignore","started":"2026-08-07T08:44:46.640413977Z"}]}
{"statuses":[{"id":"transferring context:","vertex":"sha256:9972a57f977c23ca76125b42c1c405dee759e25b0492da93b67b0c4d4b55f249","name":"transferring","current":0,"timestamp":"2026-08-07T08:44:46.642157288Z","started":"2026-08-07T08:44:46.64215638Z"}]}
{"statuses":[{"id":"transferring context:","vertex":"sha256:9972a57f977c23ca76125b42c1c405dee759e25b0492da93b67b0c4d4b55f249","name":"transferring","current":0,"timestamp":"2026-08-07T08:44:46.642649561Z","started":"2026-08-07T08:44:46.64215638Z"}]}
{"statuses":[{"id":"transferring context:","vertex":"sha256:9972a57f977c23ca76125b42c1c405dee759e25b0492da93b67b0c4d4b55f249","name":"transferring","current":2,"timestamp":"2026-08-07T08:44:46.642974544Z","started":"2026-08-07T08:44:46.64215638Z","completed":"2026-08-07T08:44:46.642974231Z"}]}
{"vertexes":[{"digest":"sha256:9972a57f977c23ca76125b42c1c405dee759e25b0492da93b67b0c4d4b55f249","name":"[internal] load .dockerignore","started":"2026-08-07T08:44:46.640413977Z","completed":"2026-08-07T08:44:46.648355222Z"}]}
{"vertexes":[{"digest":"sha256:25e23d17c54fc2311a0aac20b1c24b40752fd85a2973aa029785576bae901367","name":"[1/1] FROM quay.io/ansible/docker-test-containers:alpine3.8@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4"}]}
{"vertexes":[{"digest":"sha256:25e23d17c54fc2311a0aac20b1c24b40752fd85a2973aa029785576bae901367","name":"[1/1] FROM quay.io/ansible/docker-test-containers:alpine3.8@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4","started":"2026-08-07T08:44:46.653880579Z"}]}
{"statuses":[{"id":"resolve quay.io/ansible/docker-test-containers:alpine3.8@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4","vertex":"sha256:25e23d17c54fc2311a0aac20b1c24b40752fd85a2973aa029785576bae901367","current":0,"timestamp":"2026-08-07T08:44:46.656382309Z","started":"2026-08-07T08:44:46.656382078Z"}]}
{"vertexes":[{"digest":"sha256:25e23d17c54fc2311a0aac20b1c24b40752fd85a2973aa029785576bae901367","name":"[1/1] FROM quay.io/ansible/docker-test-containers:alpine3.8@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4","started":"2026-08-07T08:44:46.653880579Z","completed":"2026-08-07T08:44:46.667196364Z"}],"statuses":[{"id":"resolve quay.io/ansible/docker-test-containers:alpine3.8@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4","vertex":"sha256:25e23d17c54fc2311a0aac20b1c24b40752fd85a2973aa029785576bae901367","current":0,"timestamp":"2026-08-07T08:44:46.667155226Z","started":"2026-08-07T08:44:46.656382078Z","completed":"2026-08-07T08:44:46.667154942Z"}]}
{"vertexes":[{"digest":"sha256:25e23d17c54fc2311a0aac20b1c24b40752fd85a2973aa029785576bae901367","name":"[1/1] FROM quay.io/ansible/docker-test-containers:alpine3.8@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4","started":"2026-08-07T08:44:46.668014119Z","cached":true}]}
{"vertexes":[{"digest":"sha256:25e23d17c54fc2311a0aac20b1c24b40752fd85a2973aa029785576bae901367","name":"[1/1] FROM quay.io/ansible/docker-test-containers:alpine3.8@sha256:954b378c375d852eb3c63ab88978f640b4348b01c1b3456a024a81536dafbbf4","started":"2026-08-07T08:44:46.668014119Z","completed":"2026-08-07T08:44:46.668560923Z","cached":true}]}
{"vertexes":[{"digest":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","name":"exporting to image","started":"2026-08-07T08:44:46.671062871Z"}],"statuses":[{"id":"exporting layers","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.671110552Z","started":"2026-08-07T08:44:46.67111037Z"}]}
{"statuses":[{"id":"exporting layers","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.671811781Z","started":"2026-08-07T08:44:46.67111037Z","completed":"2026-08-07T08:44:46.671811384Z"}]}
{"statuses":[{"id":"exporting manifest sha256:7b4c0e5c5f3b6dcff5b5d877dc9e1f6e19d2059db8ad8eb15bcbd061a43d080c","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.671982604Z","started":"2026-08-07T08:44:46.671982389Z"}]}
{"statuses":[{"id":"exporting manifest sha256:7b4c0e5c5f3b6dcff5b5d877dc9e1f6e19d2059db8ad8eb15bcbd061a43d080c","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.67449343Z","started":"2026-08-07T08:44:46.671982389Z","completed":"2026-08-07T08:44:46.674493176Z"},{"id":"exporting config sha256:64e915d425cffa548b252ecc3f825618ef099c9a4744d5d84e5b912a5815a4b0","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.674497495Z","started":"2026-08-07T08:44:46.674497342Z"}]}
{"statuses":[{"id":"exporting config sha256:64e915d425cffa548b252ecc3f825618ef099c9a4744d5d84e5b912a5815a4b0","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.676633239Z","started":"2026-08-07T08:44:46.674497342Z","completed":"2026-08-07T08:44:46.676632886Z"}]}
{"statuses":[{"id":"exporting attestation manifest sha256:b8cb619b8fa9409b49e4f75f4c261190d58adceb2e1fb4d28b9afe8e7dc942b9","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.687846953Z","started":"2026-08-07T08:44:46.687846681Z"}]}
{"statuses":[{"id":"exporting attestation manifest sha256:b8cb619b8fa9409b49e4f75f4c261190d58adceb2e1fb4d28b9afe8e7dc942b9","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.701039943Z","started":"2026-08-07T08:44:46.687846681Z","completed":"2026-08-07T08:44:46.701039628Z"},{"id":"exporting manifest list sha256:819eaa85552e3613777fc46127aadcd71ba6a50c31afd2e08308bde30d478126","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.701061142Z","started":"2026-08-07T08:44:46.701060976Z"}]}
{"statuses":[{"id":"exporting manifest list sha256:819eaa85552e3613777fc46127aadcd71ba6a50c31afd2e08308bde30d478126","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.711998848Z","started":"2026-08-07T08:44:46.701060976Z","completed":"2026-08-07T08:44:46.711998494Z"},{"id":"naming to docker.io/library/foo:latest","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.712004429Z","started":"2026-08-07T08:44:46.71200427Z"}]}
{"statuses":[{"id":"naming to docker.io/library/foo:latest","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.714262209Z","started":"2026-08-07T08:44:46.71200427Z","completed":"2026-08-07T08:44:46.714261832Z"},{"id":"unpacking to docker.io/library/foo:latest","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.714283569Z","started":"2026-08-07T08:44:46.714281638Z"}]}
{"statuses":[{"id":"unpacking to docker.io/library/foo:latest","vertex":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","current":0,"timestamp":"2026-08-07T08:44:46.719834695Z","started":"2026-08-07T08:44:46.714281638Z","completed":"2026-08-07T08:44:46.719834407Z"}]}
{"vertexes":[{"digest":"sha256:3727fa1768102f3df504db07e5308b6dc3c01e55415cba286813aa4ffc73a733","name":"exporting to image","started":"2026-08-07T08:44:46.671062871Z","completed":"2026-08-07T08:44:46.727597997Z"}]}
{"vertexes":[{"digest":"sha256:eb3382d9c5325503f75b337c02a5d07e17ca67ed3eccbe853e5a7dfacd6a81c3","name":"resolving provenance for metadata file","started":"2026-08-07T08:44:46.789818257Z"}]}
{"vertexes":[{"digest":"sha256:eb3382d9c5325503f75b337c02a5d07e17ca67ed3eccbe853e5a7dfacd6a81c3","name":"resolving provenance for metadata file","started":"2026-08-07T08:44:46.789818257Z","completed":"2026-08-07T08:44:46.793011354Z"}]}
{"id":"Image foo","status":"Done","text":"Built"}
{"id":"Container t-foo-1","status":"Done","text":"Running"}

Here you can see that the image was built, the exported manifest has a new ID, but the container isn't recreated.

Also this clearly depends on other things as the Docker Compose and Docker daemon versions, since I have the same versions running on my local Arch box, but there the build always results in the same image ID.

Activity

  1. felixfontein
    added
    kind/bug
    status/0-triage
    on Aug 7, 2026
  2. ndeloof
    added a commit that references this issue on Aug 13, 2026

    ci: run e2e matrix with and without the containerd image store

    aedcbba
  3. glours
    added a commit that references this issue on Aug 13, 2026

    ci: run e2e matrix with and without the containerd image store

    c936d76
  4. ndeloof
    added a commit that references this issue on Aug 13, 2026

    fix(images): tolerate containers whose image record is gone

    1e7fcfd
  5. ndeloof
    removed
    status/0-triage
    on Aug 14, 2026
  6. ndeloof
    self-assigned this
    on Aug 14, 2026
  7. glours
    closed this as completedin #14028on Aug 14, 2026
  8. pull
    added a commit that references this issue on Aug 14, 2026

    fix(images): tolerate containers whose image record is gone

    df094b0

  9. felixfontein commented on Aug 17, 2026

    @felixfontein
    felixfontein
    on Aug 17, 2026
    ContributorAuthor

    Thanks for fixing this! I can confirm that this no longer happens with Docker Compose 5.5.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

kind/bug

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions