Repository navigation
docker-compose V2 fails for port range and replicas>1 where V1 would succeed #8530
Description
Description
Using multiple replicas of a container where each replica should use one of the ports selected by a port-range:
version: "3.4"
services:
dummy:
image: busybox
command: ["sh", "-c", "while true ; do sleep 3 ; date ; done"]
deploy:
replicas: 2
ports:
- "6005-6015:5000"
When starting up these containers with docker-compose V1 the container instances use port-mapping where each container picks one host-port from the selected port range to map to internal container port.
$ docker-compose ps
Name Command State Ports
----------------------------------------------------------------------------------------------------------------
docker-compose-issue_dummy_1 sh -c while true ; do slee ... Up 0.0.0.0:6007->5000/tcp,:::6007->5000/tcp
docker-compose-issue_dummy_2 sh -c while true ; do slee ... Up 0.0.0.0:6008->5000/tcp,:::6008->5000/tcp
When starting up these containers with docker-compose V2 the containers use all ports from the selected port range to map to internal container port, resulting in error like the following for the second instance of that service:
Error response from daemon: driver failed programming external connectivity on endpoint docker-compose-issue_dummy_1 (6d9cff4937b3b6946e0bb949a36362cf91477f841be34ac9bd2693453b6e8837): Bind for 0.0.0.0:6009 failed: port is already allocated
Steps to reproduce the issue:
- Switch to docker-compose V1,
docker-compose disable-v2 - Startup containers,
docker-compose up -d - List running containers and the port mapping they use,
docker-compose ps - Shutdown service,
docker-compose down - Switch to docker-compose V2,
docker-compose enable-v2 - Startup containers,
docker-compose up -d - docker-compose fails to startup because of port mapping clash
Describe the results you received:
docker-compose.yaml only seems to work for V1 and not V2.
Describe the results you expected:
I would expect that docker-compose.yaml can be run by V1 and V2 alike.
Output of docker version:
Client:
Cloud integration: 1.0.17
Version: 20.10.7
API version: 1.41
Go version: go1.16.4
Git commit: f0df350
Built: Wed Jun 2 11:56:22 2021
OS/Arch: darwin/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.7
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: b0f5bc3
Built: Wed Jun 2 11:54:58 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.6
GitCommit: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc:
Version: 1.0.0-rc95
GitCommit: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Output of docker info:
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
compose: Docker Compose (Docker Inc., 2.0.0-beta.3)
scan: Docker Scan (Docker Inc., v0.8.0)
Server:
Containers: 30
Running: 10
Paused: 0
Stopped: 20
Images: 136
Server Version: 20.10.7
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.25-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 3
Total Memory: 11.7GiB
Name: docker-desktop
ID: 65BF:GFI6:S3ZY:URLI:EHO5:GFN5:PT7Y:Q7JW:ACRV:FJCE:YM5X:2YFM
Docker Root Dir: /var/lib/docker
Debug Mode: true
File Descriptors: 159
Goroutines: 157
System Time: 2021-06-24T14:52:06.2642643Z
EventsListeners: 5
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional environment details (AWS ECS, Azure ACI, local, etc.):
local, MacOS BigSur 11.4
Activity
manuel-koch commentedon Jun 24, 2021
When running the example from above the V1 docker-compose will issue a warning but proceed without an error:
WARNING: The "dummy" service specifies a port on the host. If multiple containers for this service are created on a single host, the port will clash.Is it possible that V2 is more restricted and hence just fails instead of reporting a warning like V1 ?
Additionally the docker-compose ports documentation is not very clear about this apparently working port-range feature in V1, i.e. automatically assign a different port from the given range of ports to multi-replica-services.
ndeloof commentedon Jun 28, 2021
V2 indeed doesn't reproduce this V1 warning, this is something we should consider.
Compose v1 creates containers sequentially, as an equivalent to
docker run. Then the port specification is passed to engine. According to https://docs.docker.com/engine/reference/run/#expose-incoming-ports:"When specifying a range for hostPort only, the
containerPort must not be a range. In this case the
container port is published somewhere within the
specified hostPort range. (e.g.,
-p 1234-1236:1234/tcp)".Compose v2 creates containers in parallel, so this engine feature doesn't apply and this results in a port conflict. Seems we will have to drop this (premature?) optimization to restore this feature.
infor7 commentedon Oct 22, 2021
Hi
I have same problem on V2. The host ports are assigned to one container instead of distributing them among all replicas.
On V1 replicas works just fine:
seonglae commentedon Nov 5, 2021
This is still problem
ndeloof commentedon Nov 5, 2021
ran a quick test, and this actually isn't an issue with replicas. Even using
replicas: 1container is bound to the whole range, not "somrwhere within the specified range`.. so there's something wrong with compose v2 passing port specification to the engine.
ndeloof commentedon Nov 5, 2021
confirmed.
compose-v1 creates container as:
compose-v2 creates container as:
ndeloof commentedon Nov 5, 2021
the issue is in compose-go, which "extends"
6005-6015:5000into a list of bindings for each and every values in the rangeC-Powers commentedon Nov 16, 2021
I'm not sure if this is the same problem, but researching how to solve my own problem took me here. I was having a similar problem on a Mac,
and I was able to fix it by checking:
and then turning off the
Airplay ReceiverService inSystem Preferences -> Sharingndeloof commentedon Nov 17, 2021
compose-go both extends port range to a list of ports, but also assumes this is expected
and types are designed to only accept an actual port (
uint32) not range.I guess we are stuck here :'(
ndeloof commentedon Nov 17, 2021
This will also require some clarification on https://github.com/compose-spec/compose-spec/blob/master/spec.md#ports if we want this to be addressed
publishedport value (https://github.com/compose-spec/compose-spec/blob/master/schema/compose-spec.json#L321)Hazmi35 commentedon Dec 31, 2021
Sorry for the bump, any updates?
rs-renato commentedon Dec 31, 2021
the same issue here, I force myself to downgrade to v1
hacdias commentedon Mar 10, 2022
Is there a known workaround for this?
1 remaining item
flipkickmedia commentedon Mar 23, 2022
Docker version 20.10.13, build a224086
Docker Compose version v2.3.3
I'm seeing this issue also.
@hacdias @Hazmi35 If you run the
docker compose upseveral times (one for each replica) you can eventually get the system running with the replicas and port mappings.Currently having to remove the replica config an repeat the container instance config for each replica I need.
This issue needs looking at...its been open for a very long time with a significant bug in the use of docker-compose when testing a scaling setup. I've moved to Kubernetes/skaffold.
scottbelden commentedon May 2, 2022
$ docker --version
Docker version 20.10.14, build a224086
$ docker-compose --version
Docker Compose version v2.4.1
Also running into this issues. Sometimes it will bring everything up correctly, and other times I will get the bind error and one of the replicas will not start. I use Docker for Mac so I don't know if that is part of the reason.
Inushin commentedon Jun 21, 2022
If I am not mistaken, the "bug" is still here.
spiritedsnowcat commentedon Jul 6, 2022
Same issue as #7188.
Port range not longer works on docker-compose 2.2.2 and higher, appar…
ajskateboarder commentedon Aug 7, 2022
I have a similar problem as well.
I am trying to create at least 2 instances of the same image and it fails miserably:
I referenced this in an issue - #9727
sespinoza-dev commentedon Oct 10, 2022
I think this issue was resolved in the new version.
This configuration worked for me.
Docker Compose Version.
Configuration file
docker-compose.yml.And the replicas it created:
All the replicas correctly mapped to
4060,4061, and4062.sh-cho commentedon Nov 9, 2022
It's still happening. (m1 mac, macOS 12.1 monterey)
I'd like to fix this, but don't know where to begin. Any advice?
alexanderwink commentedon Nov 9, 2022
It seems to be more of a timing issue afaik. I have seen the exact same docker-compose work if there is a delay (in my case due to high cpu load) so that the first container starts and it takes a second or two before the second container starts. When there is almost no delay (just some milliseconds) I get the bind-error.
fritzdimmel commentedon Dec 7, 2022
@sh-cho Having the same issue, docker-compose v 2.6.0 on Mac M1.
Only way I could run multiple instances is by removing the port range
(e.g. I had:
ports: - "11001-11004:7777"). I changed this to:
ports: - "7777"and then my 4 instances could be started.
Unfortunately with random external ports, but at least they could be started.
Had exactly the same error message:
Error response from daemon: Ports are not available: exposing port TCP 0.0.0.0:11001 -> 0.0.0.0:0: listen tcp 0.0.0.0:11001: bind: address already in useLooks like it tries to assign to "internal port 0" which is quite odd.
Cheers
ndeloof commentedon Dec 7, 2022
AFAICT we need to make the
ContainerCreateAPI calls sequential to enforce a distinct port is assignedalexanderwink commentedon Dec 13, 2022
Great work!
EParisot commentedon Feb 6, 2023
Hi, I keep facing this issue with docker compose 2.15.1 with following setting
running
docker compose upstarts the first instance but shows the error message (Error response from daemon: Ports are not available: exposing port TCP 0.0.0.0:8081 -> 0.0.0.0:0: listen tcp 0.0.0.0:8081: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.), running it a second time make my 2 replicas working as expected (confirmed withdocker ps)