docker-compose CLI output is not "greppable" any more

[niftyhack]

• I have tried with the latest version of Docker Desktop
• I have tried disabling enabled experimental features
• I have uploaded Diagnostics
• Diagnostics ID: F422FCD1-9103-499F-B66B-A21B08AF9789/20220218084251

Expected behavior

CLI output from docker-compose is "greppable".

Actual behavior

CLI output from docker-compose is not "greppable". But CLI output from docker command is "greppable".

Information

• Problem is reproducible
• Problem was noticed yesterday afternoon. Yesterday evening Docker Desktoop was updated, problem persists
• Problem appeared with an update, but I don't know since when it exists
• macOS Version: Big Sur 11.6.2
• Intel chip: 2,7 GHz Quad-Core Intel Core i7
• Docker Desktop Version: 4.5.0 (74594)

Output of /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check

$ /Applications/Docker.app/Contents/MacOS/com.docker.diagnose check
Starting diagnostics

[PASS] DD0027: is there available disk space on the host?
[PASS] DD0028: is there available VM disk space?
[PASS] DD0031: does the Docker API work?
[PASS] DD0004: is the Docker engine running?
[PASS] DD0011: are the LinuxKit services running?
[PASS] DD0016: is the LinuxKit VM running?
[PASS] DD0001: is the application running?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0017: can a VM be started?
[PASS] DD0015: are the binary symlinks installed?
[PASS] DD0003: is the Docker CLI working?
[PASS] DD0013: is the $PATH ok?
[PASS] DD0007: is the backend responding?
[PASS] DD0014: are the backend processes running?
[PASS] DD0008: is the native API responding?
[PASS] DD0009: is the vpnkit API responding?
[PASS] DD0010: is the Docker API proxy responding?
[PASS] DD0012: is the VM networking working?
[PASS] DD0032: do Docker networks overlap with host IPs?
[SKIP] DD0030: is the image access management authorized?
[PASS] DD0019: is the com.docker.vmnetd process responding?
[PASS] DD0033: does the host have Internet access?
No fatal errors detected.

Steps to reproduce the behavior

$ cat docker-compose.yml
services:
  web:
    image: nginx

$ docker-compose up -d
[+] Running 1/1
 ⠿ Container tmp-web-1  Started

$ docker ps
CONTAINER ID   IMAGE     COMMAND                  CREATED          STATUS          PORTS     NAMES
2904a5bfd2ed   nginx     "/docker-entrypoint.…"   49 seconds ago   Up 49 seconds   80/tcp    tmp-web-1

$ docker exec 2904a5bfd2ed ls -lh
total 80K
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 bin
drwxr-xr-x   2 root root 4.0K Dec 11 17:25 boot
drwxr-xr-x   5 root root  340 Feb 18 08:36 dev
drwxr-xr-x   1 root root 4.0K Jan 26 08:58 docker-entrypoint.d
-rwxrwxr-x   1 root root 1.2K Jan 26 08:58 docker-entrypoint.sh
drwxr-xr-x   1 root root 4.0K Feb 18 08:36 etc
drwxr-xr-x   2 root root 4.0K Dec 11 17:25 home
drwxr-xr-x   1 root root 4.0K Jan 25 00:00 lib
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 lib64
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 media
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 mnt
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 opt
dr-xr-xr-x 199 root root    0 Feb 18 08:36 proc
drwx------   2 root root 4.0K Jan 25 00:00 root
drwxr-xr-x   1 root root 4.0K Feb 18 08:36 run
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 sbin
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 srv
dr-xr-xr-x  13 root root    0 Feb 18 08:36 sys
drwxrwxrwt   1 root root 4.0K Jan 26 08:58 tmp
drwxr-xr-x   1 root root 4.0K Jan 25 00:00 usr
drwxr-xr-x   1 root root 4.0K Jan 25 00:00 var

$ docker exec 2904a5bfd2ed ls -lh | grep boot ######### <- This work's.
drwxr-xr-x   2 root root 4.0K Dec 11 17:25 boot

$ docker-compose exec web ls -lh
total 80K
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 bin
drwxr-xr-x   2 root root 4.0K Dec 11 17:25 boot
drwxr-xr-x   5 root root  340 Feb 18 08:36 dev
drwxr-xr-x   1 root root 4.0K Jan 26 08:58 docker-entrypoint.d
-rwxrwxr-x   1 root root 1.2K Jan 26 08:58 docker-entrypoint.sh
drwxr-xr-x   1 root root 4.0K Feb 18 08:36 etc
drwxr-xr-x   2 root root 4.0K Dec 11 17:25 home
drwxr-xr-x   1 root root 4.0K Jan 25 00:00 lib
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 lib64
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 media
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 mnt
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 opt
dr-xr-xr-x 199 root root    0 Feb 18 08:36 proc
drwx------   2 root root 4.0K Jan 25 00:00 root
drwxr-xr-x   1 root root 4.0K Feb 18 08:36 run
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 sbin
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 srv
dr-xr-xr-x  13 root root    0 Feb 18 08:36 sys
drwxrwxrwt   1 root root 4.0K Jan 26 08:58 tmp
drwxr-xr-x   1 root root 4.0K Jan 25 00:00 usr
drwxr-xr-x   1 root root 4.0K Jan 25 00:00 var

$ docker-compose exec web ls -lh | grep boot ######### -> This doesn't work.
total 80K
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 bin
drwxr-xr-x   2 root root 4.0K Dec 11 17:25 boot
drwxr-xr-x   5 root root  340 Feb 18 08:36 dev
drwxr-xr-x   1 root root 4.0K Jan 26 08:58 docker-entrypoint.d
-rwxrwxr-x   1 root root 1.2K Jan 26 08:58 docker-entrypoint.sh
drwxr-xr-x   1 root root 4.0K Feb 18 08:36 etc
drwxr-xr-x   2 root root 4.0K Dec 11 17:25 home
drwxr-xr-x   1 root root 4.0K Jan 25 00:00 lib
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 lib64
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 media
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 mnt
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 opt
dr-xr-xr-x 199 root root    0 Feb 18 08:36 proc
drwx------   2 root root 4.0K Jan 25 00:00 root
drwxr-xr-x   1 root root 4.0K Feb 18 08:36 run
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 sbin
drwxr-xr-x   2 root root 4.0K Jan 25 00:00 srv
dr-xr-xr-x  13 root root    0 Feb 18 08:36 sys
drwxrwxrwt   1 root root 4.0K Jan 26 08:58 tmp
drwxr-xr-x   1 root root 4.0K Jan 25 00:00 usr
drwxr-xr-x   1 root root 4.0K Jan 25 00:00 var     

[glours]

Can you share the output of the following command docker info please?

[ndeloof]

Can you please clarify what you expect compose output to look like so it is "greppable"?
I.e what's the command you'd like to type that would include use of grep?

[ndeloof]

eventually understood the issue during a chat session with @glours :)
so, basically, before 2.2.3 we had code to detect current process is running or not within a TTY, and then adjust the way we interact with stdin/stdout. This had some complex side effects for users who want to redirect some stream to stdin, etc.
In 2.2.3 we made this user responsibility to disable TTY (--no-tty) when they want to rely on POSIX streams, typically to pipe output to another command. This is the issue you have here.
A long term solution is for us to share more code with docker/cli, so that we don't need to reinvent the wheel and suffer from issues that docker command line already addressed. see docker/cli#3436

[thaJeztah]

Yes, it's a bit unfortunate that the defaults (for TTY (-t / --tty) are "flipped" (compose enables it by default); while there's things to say for both approaches (compose focusses more on interactively running things, whereas the docker engine / cli considers that "opt-in"), it makes things somewhat inconsistent (and sometimes quite "subtle").

Curious now; what's the default on compose for attaching stdin (-i, --interactive) ? I guess default is "enabled" as well?

I actually recalled I wanted to write a proposal that's slightly related, and just did: #9207

[niftyhack]

@glours

$ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.7.1)
  compose: Docker Compose (Docker Inc., v2.2.3)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 73
  Running: 10
  Paused: 0
  Stopped: 63
 Images: 71
 Server Version: 20.10.12
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.76-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.774GiB
 Name: docker-desktop
 ID: EIFS:XU3N:Y7XW:SWSC:VVAI:JPH5:PTGM:LBST:CIBL:WC4U:Q4Y7:IMVX
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 270
  Goroutines: 151
  System Time: 2022-02-25T11:11:06.0058852Z
  EventsListeners: 4
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false

[niftyhack]

Can you please clarify what you expect compose output to look like so it is "greppable"? I.e what's the command you'd like to type that would include use of grep?

Well, using the docker-compose.yml from the provided example ...

docker exec 2904a5bfd2ed ls -lh | grep boot

... greps the output from the docker command for boot and displays the lines containing boot.

However:

docker-compose exec web ls -lh | grep boot

does not seem to grep correctly. The output I get is the whole output from the ls -lh command from within the container.

[thaJeztah]

@niftyhack and it does work when adding the -T option to the docker compose exec ?

[niftyhack]

@thaJeztah

Somehow, but the output is mangled:

$ docker-compose exec -T web ls -lh | grep boot
drwxr-xr-x   2 root root 4.0K Dec 11 17:25 boot
                                               ~/tmp/docker-compose
$

~/tmp/docker-compose is part of my prompt, which display the current working directory on an extra line.

You can see it better if you pass --help as an option for ls in the container and grep the output of ls --help:

$ docker-compose exec -T web ls --help | grep sort
Sort entries alphabetically if none of -cftuvSUX nor --sort is specified.
                                                                           -c                         with -lt: sort by, and show, ctime (time of last
                                                                                                                                                                                     with -l: show ctime and sort by name;
                                                                                                                                                                                                                                                         otherwise: sort by ctime, newest first
                   -f                         do not sort, enable -aU, disable -ls --color
                                                                                                                         can be augmented with a --sort option, but any
                                                                                                                                                                                                      use of --sort=none (-U) disables grouping
                                                                                                                                                                                                                                                 -r, --reverse              reverse order while sorting
                           -S                         sort by file size, largest first
                                                                                            --sort=WORD            sort by WORD instead of name: none (-U), size (-S),
                                                                                                                                                                                                   with --sort=time, sort by WORD (newest first)
                                                                                                                                                                                                                                                  -t                         sort by time, newest first; see --time
                                       -u                         with -lt: sort by, and show, access time;
                                                                                                                                          with -l: show access time and sort by name;
                                                                                                                                                                                                                    otherwise: sort by access time, newest first
                                                                                                                                                                                                                                                                  -U                         do not sort; list entries in directory order
                                                             -v                         natural sort of (version) numbers within text
                                                                                                                                       -X                         sort alphabetically by entry extension

The output is also mangled if I just do docker-compose exec -T web ls --help, but somehow I can't paste the whole output to Github. On the other hand, piping the output to vim is fine, docker-compose exec -T web ls --help | vim - displays ls --help nicely formatted in a vim buffer.

I also noticed that docker-compose help does not mention the -T option.

[chuck-confluent]

This broke a demo of mine so it either works for users with the latest version of docker-compose, or users with older versions, but not both. Let me give an example.

Me, with a newer version of docker compose (2.2.3):

gitpod /workspace/cp-demo (DEVX-2720) $ docker-compose exec --no-TTY connect curl -X GET --cert /etc/kafka/secrets/connect.certificate.pem --key /etc/kafka/secrets/connect.key --tlsv1.2 --cacert /etc/kafka/secrets/snakeoil-ca-1.crt -u superUser:superUser https://connect:8083/connectors/wikipedia-sse/status 2>/dev/null | jq -r .connector.state

gives output RUNNING

Friend, running the same command with an older version (1.29) of docker-compose gives blank output.

Conversely, friend runs without --no-TTY:

docker-compose exec connect curl -X GET --cert /etc/kafka/secrets/connect.certificate.pem --key /etc/kafka/secrets/connect.key --tlsv1.2 --cacert /etc/kafka/secrets/snakeoil-ca-1.crt -u superUser:superUser https://connect:8083/connectors/$connectorName/status 2>/dev/null | jq -r .connector.state 
RUNNING

Whereas I get blank output.

[glours]

@chuck-confluent which version of Docker Compose are you using? Can you give a try to the v2.4.1 version and let us now if it's work as expected please?