Installation fails

Our CI workflow install the diff plugin. Since an hour ago it start failing (commit 4790fb1)
We are running:

helm plugin install https://github.com/databus23/helm-diff

Error:

awk: cmd. line:1: warning: regexp escape sequence `"' is not a known regexp operator
Downloading
curl: (3) URL using bad/illegal format or missing URL
Failed to install helm-diff
For support, go to https://github.com/databus23/helm-diff.
Error: plugin install hook for "diff" exited with error
Error: Process completed with exit code 1.

I'm seeing this as well

It works fine when you run it with specific version:
helm plugin install https://github.com/databus23/helm-diff --version 3.1.3

I believe it is that latest commit from #267
Running helm plugin install https://github.com/databus23/helm-diff locally fails.
Whereas helm plugin install https://github.com/databus23/helm-diff --version 97f77494396694beb878128430205888a91d70b4 succeeds (that is the second to last commit)

I think the grep $ARCH from

helm-diff/install-binary.sh

Lines 80 to 84 in 4790fb1

    
           if type "curl" >/dev/null; then 
        
             DOWNLOAD_URL=$(curl -s $url | grep $OS | grep $ARCH | awk '/\"browser_download_url\":/{gsub( /[,\"]/,"", $2); print $2}') 
        
           elif type "wget" >/dev/null; then 
        
             DOWNLOAD_URL=$(wget -q -O - $url | grep $OS | grep $ARCH | awk '/\"browser_download_url\":/{gsub( /[,\"]/,"", $2); print $2}') 
        
           fi

is returning failure exit code because the latest release (3.1.3) doesn't have architecture specific artifacts (only OS specific) and hence DOWNLOAD_URL is empty after getDownloadURL call

pinning helm diff version, workaround for databus23/helm-diff#322

Seems need a new release

pin helm-diff version to work around broken master

Just added arch-specific archives to the existing v3.1.3 release. I hope those fix the issue https://github.com/databus23/helm-diff/releases/tag/v3.1.3

@mumoshu

Thanks @mumoshu to ensure I had a clean environment, I ran helm plugin uninstall diff; sudo rm -r ~/Library/Caches/helm/plugins/https-github.com-databus23-helm-diff; helm plugin install https://github.com/databus23/helm-diff and everything installed locally as expected :)

@damienpontifex

@damienpontifex Glad to hear it worked! Thanks a lot for your confirmation ☺️

pinning helm diff version, workaround for databus23/helm-diff#322 (#4)

Just added arch-specific archives to the existing v3.1.3 release. I hope those fix the issue https://github.com/databus23/helm-diff/releases/tag/v3.1.3

This now causes new issues when using a commit hash as version, where it has a download url consisting of multiple urls;

$ helm plugin install --debug https://github.com/databus23/helm-diff --version 97f77494396694beb878128430205888a91d70b4
[debug] vcs_installer.go:162: updating https://github.com/databus23/helm-diff
[debug] vcs_installer.go:152: setting version to "97f77494396694beb878128430205888a91d70b4"
[debug] vcs_installer.go:91: copying /Users/koenpunt/Library/Caches/helm/plugins/https-github.com-databus23-helm-diff to /Users/koenpunt/Library/helm/plugins/helm-diff
plugin_install.go:82: [debug] loading plugin from /Users/koenpunt/Library/helm/plugins/helm-diff
plugin.go:60: [debug] running install hook: /bin/sh -c $HELM_PLUGIN_DIR/install-binary.sh
Downloading https://github.com/databus23/helm-diff/releases/download/v3.1.3/helm-diff-macos-amd64.tgz
https://github.com/databus23/helm-diff/releases/download/v3.1.3/helm-diff-macos-arm64.tgz
https://github.com/databus23/helm-diff/releases/download/v3.1.3/helm-diff-macos.tgz
curl: (3) URL using bad/illegal format or missing URL
Failed to install helm-diff
	For support, go to https://github.com/databus23/helm-diff.
Error: plugin install hook for "diff" exited with error
helm.go:88: [debug] plugin install hook for "diff" exited with error
main.runHook
	helm.sh/helm/v3/cmd/helm/plugin.go:67
main.(*pluginInstallOptions).run
	helm.sh/helm/v3/cmd/helm/plugin_install.go:88
main.newPluginInstallCmd.func3
	helm.sh/helm/v3/cmd/helm/plugin_install.go:59
github.com/spf13/cobra.(*Command).execute
	github.com/spf13/cobra@v1.2.1/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
	github.com/spf13/cobra@v1.2.1/command.go:974
github.com/spf13/cobra.(*Command).Execute
	github.com/spf13/cobra@v1.2.1/command.go:902
main.main
	helm.sh/helm/v3/cmd/helm/helm.go:87
runtime.main
	runtime/proc.go:255
runtime.goexit
	runtime/asm_arm64.s:1133

@koenpunt

@koenpunt Thanks for reporting! In that case you can just replace 97f77494396694beb878128430205888a91d70b4 with v3.1.3 in the helm plugin install command, as it's really the same for helm-diff.

Specifying 97f77494396694beb878128430205888a91d70b4 results in grabbing plugin.yaml for that commit, which may result in downloading the version of helm-diff that corresponds to the version number embedded in the plugin.yaml, which is 3.1.3 in this case.

helm-diff/plugin.yaml

Line 4 in 97f7749

version: "3.1.3"

In that case you can just replace 97f77494396694beb878128430205888a91d70b4 with v3.1.3

I did that for now, but it should be possible to use a single commit as well? Or at least the script should be able to resolve to a single download url I suppose?

@mumoshu

@mumoshu the installation is failing for us.... we are installing in a docker container based on openshift/jenkins-slave-base-centos7

this is the run command:

RUN cd /tmp \
    && mkdir -p /tmp/helm \
    && curl -LO https://get.helm.sh/helm-v3.4.1-linux-amd64.tar.gz \
    && tar -zxvf helm-v3.4.1-linux-amd64.tar.gz -C /tmp/helm \
    && mv /tmp/helm/linux-amd64/helm /usr/local/bin/helm \
    && chmod a+x /usr/local/bin/helm \
    && helm version \
    && helm plugin install --debug https://github.com/databus23/helm-diff --version v3.1.3 \
    && helm plugin install --debug https://github.com/jkroepke/helm-secrets --version v3.3.5 \
    && sops --version \
    && rm -rf /tmp/helm

these are the logs:

Step 16/26 : RUN cd /tmp     && mkdir -p /tmp/helm     && curl -LO https://get.helm.sh/helm-v3.4.1-linux-amd64.tar.gz     && tar -zxvf helm-v3.4.1-linux-amd64.tar.gz -C /tmp/helm     && mv /tmp/helm/linux-amd64/helm /usr/local/bin/helm     && chmod a+x /usr/local/bin/helm     && helm version     && helm plugin install --debug https://github.com/databus23/helm-diff --version v3.1.3     && helm plugin install --debug https://github.com/jkroepke/helm-secrets --version v3.3.5     && sops --version     && rm -rf /tmp/helm
 ---> Running in e7563534409d
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 12.7M  100 12.7M    0     0  22.6M      0 --:--:-- --:--:-- --:--:-- 22.7M
linux-amd64/
linux-amd64/LICENSE
linux-amd64/README.md
linux-amd64/helm
W0107 15:56:17.894930      15 loader.go:223] Config not found: /var/lib/origin/openshift.local.config/master/admin.kubeconfig
version.BuildInfo{Version:"v3.4.1", GitCommit:"c4e74854886b2efe3321e185578e6db9be0a6e29", GitTreeState:"clean", GoVersion:"go1.14.11"}
W0107 15:56:17.944550      26 loader.go:223] Config not found: /var/lib/origin/openshift.local.config/master/admin.kubeconfig
[debug] vcs_installer.go:159: cloning https://github.com/databus23/helm-diff to /home/jenkins/.cache/helm/plugins/https-github.com-databus23-helm-diff
[debug] vcs_installer.go:152: setting version to "v3.1.3"
[debug] vcs_installer.go:91: copying /home/jenkins/.cache/helm/plugins/https-github.com-databus23-helm-diff to /home/jenkins/.local/share/helm/plugins/helm-diff
plugin_install.go:82: [debug] loading plugin from /home/jenkins/.local/share/helm/plugins/helm-diff
plugin.go:61: [debug] running install hook: /usr/bin/sh -c $HELM_PLUGIN_DIR/install-binary.sh
W0107 15:56:19.034371      26 loader.go:223] Config not found: /var/lib/origin/openshift.local.config/master/admin.kubeconfig
W0107 15:56:19.084887     145 loader.go:223] Config not found: /var/lib/origin/openshift.local.config/master/admin.kubeconfig
Downloading https://github.com/databus23/helm-diff/releases/download/v3.1.3/helm-diff-linux-amd64.tgz
https://github.com/databus23/helm-diff/releases/download/v3.1.3/helm-diff-linux-arm64.tgz
https://github.com/databus23/helm-diff/releases/download/v3.1.3/helm-diff-linux.tgz
curl: (3) Illegal characters found in URL
Failed to install helm-diff
	For support, go to https://github.com/databus23/helm-diff.
Error: plugin install hook for "diff" exited with error
helm.go:81: [debug] plugin install hook for "diff" exited with error
main.runHook
	/home/circleci/helm.sh/helm/cmd/helm/plugin.go:68
main.(*pluginInstallOptions).run
	/home/circleci/helm.sh/helm/cmd/helm/plugin_install.go:88
main.newPluginInstallCmd.func3
	/home/circleci/helm.sh/helm/cmd/helm/plugin_install.go:59
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:842
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:950
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/cobra@v1.0.0/command.go:887
main.main
	/home/circleci/helm.sh/helm/cmd/helm/helm.go:80
runtime.main
	/usr/local/go/src/runtime/proc.go:203
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1373
The command '/bin/bash -o pipefail -c cd /tmp     && mkdir -p /tmp/helm     && curl -LO https://get.helm.sh/helm-v3.4.1-linux-amd64.tar.gz     && tar -zxvf helm-v3.4.1-linux-amd64.tar.gz -C /tmp/helm     && mv /tmp/helm/linux-amd64/helm /usr/local/bin/helm     && chmod a+x /usr/local/bin/helm     && helm version     && helm plugin install --debug https://github.com/databus23/helm-diff --version v3.1.3     && helm plugin install --debug https://github.com/jkroepke/helm-secrets --version v3.3.5     && sops --version     && rm -rf /tmp/helm' returned a non-zero code: 1

The below log implies that helm is using outdated install-binary.sh of helm-diff (which doesn't have support for new arch-specific archives available since 3.2.0) to install 3.2.0. AFAIK, that cannot happen without a corrupted/outdated plugin cache.

[2022-01-09T02:37:58.070Z] �[0;32m    amazon-ebs: Downloading https://github.com/databus23/helm-diff/releases/download/v3.2.0/helm-diff-linux-amd64.tgz�[0m
[2022-01-09T02:37:58.070Z] �[0;32m    amazon-ebs: https://github.com/databus23/helm-diff/releases/download/v3.2.0/helm-diff-linux-arm64.tgz�[0m

See HELM_CACHE_HOME value in the helm env output. If you're configuring your CI system to cache that path, it can break like that.

In case it turned out to be outdated plugin cache issue- our install-binary.sh is only backward-compatible(not forward-compatible) so this kind of issue is unavoidable. In other words, if you do want a real solid helm plugin installation step in your CI, I'd say you shouldn't cache ${HELM_CACHE_HOME}/plugins.

ok deleting with rm -rf $HOME/.cache/helm/plugins/https-github.com-databus23-helm-diff || true does not help ... :(

from helm env

HELM_BIN="helm"
HELM_CACHE_HOME="/home/jenkins/.cache/helm"
HELM_CONFIG_HOME="/home/jenkins/.config/helm"
HELM_DATA_HOME="/home/jenkins/.local/share/helm"
HELM_DEBUG="false"
HELM_KUBEAPISERVER=""
HELM_KUBEASGROUPS=""
HELM_KUBEASUSER=""
HELM_KUBECONTEXT=""
HELM_KUBETOKEN=""
HELM_MAX_HISTORY="10"
HELM_NAMESPACE="default"
HELM_PLUGINS="/home/jenkins/.local/share/helm/plugins"
HELM_REGISTRY_CONFIG="/home/jenkins/.config/helm/registry.json"
HELM_REPOSITORY_CACHE="/home/jenkins/.cache/helm/repository"
HELM_REPOSITORY_CONFIG="/home/jenkins/.config/helm/repositories.yaml"

@clemensutschig

@clemensutschig Could you try running git show --oneline --stat in both $HOME/.cache/helm/plugins/https-github.com-databus23-helm-diff and $HOME/.local/share/helm/plugins/helm-diff/ respectively after the helm plugin installation failure to see what's happening?

I'm currently using Helm 3.5.3 so try upgrading helm if that makes difference.

Please see the relevant part of install-binary.sh across versions below:

`--version 3.1.3`

helm-diff/install-binary.sh

Lines 72 to 83 in b80187e

    
           version=$(git -C "$HELM_PLUGIN_DIR" describe --tags --exact-match 2>/dev/null || :) 
        
           if [ -n "$version" ]; then 
        
             DOWNLOAD_URL="/$PROJECT_GH/releases/download/$version/helm-diff-$OS.tgz" 
        
           else 
        
             # Use the GitHub API to find the download url for this project. 
        
             url="https://api.github.com/repos/$PROJECT_GH/releases/latest" 
        
             if type "curl" >/dev/null; then 
        
               DOWNLOAD_URL=$(curl -s $url | grep $OS | awk '/\"browser_download_url\":/{gsub( /[,\"]/,"", $2); print $2}') 
        
             elif type "wget" >/dev/null; then 
        
               DOWNLOAD_URL=$(wget -q -O - $url | grep $OS | awk '/\"browser_download_url\":/{gsub( /[,\"]/,"", $2); print $2}') 
        
             fi 
        
           fi

installs 3.1.3 by grep $OS | grep $ARCH + https://github.com/databus23/helm-diff/releases/tag/v3.1.3

`--version 3.2.0`

helm-diff/install-binary.sh

Lines 72 to 83 in 860f354

    
           version=$(git -C "$HELM_PLUGIN_DIR" describe --tags --exact-match 2>/dev/null || :) 
        
           if [ -n "$version" ]; then 
        
             DOWNLOAD_URL="/$PROJECT_GH/releases/download/$version/helm-diff-$OS-$ARCH.tgz" 
        
           else 
        
             # Use the GitHub API to find the download url for this project. 
        
             url="https://api.github.com/repos/$PROJECT_GH/releases/latest" 
        
             if type "curl" >/dev/null; then 
        
               DOWNLOAD_URL=$(curl -s $url | grep $OS | grep $ARCH | awk '/\"browser_download_url\":/{gsub( /[,\"]/,"", $2); print $2}') 
        
             elif type "wget" >/dev/null; then 
        
               DOWNLOAD_URL=$(wget -q -O - $url | grep $OS | grep $ARCH | awk '/\"browser_download_url\":/{gsub( /[,\"]/,"", $2); print $2}') 
        
             fi 
        
           fi

installs 3.2.0 by grep $OS | grep $ARCH + https://github.com/databus23/helm-diff/releases/tag/v3.2.0

without `--version`

It will use the script at the HEAD of the master branch

helm-diff/install-binary.sh

Lines 85 to 89 in 7486472

    
           version=$(git -C "$HELM_PLUGIN_DIR" describe --tags --exact-match 2>/dev/null || :) 
        
           if [ "$SCRIPT_MODE" = "install" -a -n "$version" ]; then 
        
             DOWNLOAD_URL="/$PROJECT_GH/releases/download/$version/helm-diff-$OS-$ARCH.tgz" 
        
           else 
        
             DOWNLOAD_URL="/$PROJECT_GH/releases/latest/download/helm-diff-$OS-$ARCH.tgz"

installs 3.2.0 today thanks to L89

@clemensutschig

So I did some more debugging around this.

The error mentioned by @clemensutschig happens with Helm 3.4.2. Updating Helm to 3.5.3 and helm-diff to 3.2.0 fixes the problem. I am not sure what change in Helm 3.5.3 is responsible for this, but I think we can update to 3.5.3.

However, the install displays that it doesn't install version 3.2.0, but rather 3.3.1. I investigated this as it simply seems to install the latest version, no matter what is passed via --version v<VERSION>. I realised that the install script is actually broken until #307. The issue was the following line:

: ${HELM_PLUGIN_DIR:="$("${HELM_BIN}" home --debug=false)/plugins/helm-diff"}

home is not a valid Helm command (neither in the 3.5.x line nor in the latest 3.7.2). Because the script does not exit on error, it continues, but with HELM_PLUGIN_DIR being /plugins/helm-diff. Because that path doesn't exist, version=$(git -C "$HELM_PLUGIN_DIR" describe --tags --exact-match 2>/dev/null || :) returns nothing, which leads to the latest version being downloaded.

The latest version of helm-diff, 3.3.1, works correctly, however only in systems which support which, which is not POSIX compatible (see e.g. https://stackoverflow.com/questions/592620/how-can-i-check-if-a-program-exists-from-a-bash-script) and consequently fails on UBI images for example.

I'm going to provide a fix for this in a sec.

which is not POSIX compatible

😱

Merge pull request #339 from michaelsauter/fix/install-script

3.3.2 has been released with the fix. Would you mind trying once again?

@mumoshu

@mumoshu It works now, thank you very much!

@michaelsauter

@michaelsauter Thank you for your help!

@michaelsauter

@michaelsauter Thank you for spotting that, I am the one who has included the line using which, I didn't thought about the POSIX compatibility.

On the other hand the home command was already in the script, and I kept it for back compatibility with helm2.

Not sure it would still work for helm2 without that line.

@rkeidar

@rkeidar I hope it's already fixed for you too and you're having happy days.

Closing as resolved.
Thank you everyone for your support!

P.S. I'm willing to add Alpine Linux and Busybox to our test matrix (#347). Your contribution is much appreciated!

Navigation Menu

Repository navigation

Installation fails #322

14 remaining items

`--version 3.1.3`

`--version 3.2.0`

without `--version`

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	if type "curl" >/dev/null; then
	DOWNLOAD_URL=$(curl -s $url \| grep $OS \| grep $ARCH \| awk '/\"browser_download_url\":/{gsub( /[,\"]/,"", $2); print $2}')
	elif type "wget" >/dev/null; then
	DOWNLOAD_URL=$(wget -q -O - $url \| grep $OS \| grep $ARCH \| awk '/\"browser_download_url\":/{gsub( /[,\"]/,"", $2); print $2}')
	fi

	version=$(git -C "$HELM_PLUGIN_DIR" describe --tags --exact-match 2>/dev/null \|\| :)
	if [ -n "$version" ]; then
	DOWNLOAD_URL="/$PROJECT_GH/releases/download/$version/helm-diff-$OS.tgz"
	else
	# Use the GitHub API to find the download url for this project.
	url="https://api.github.com/repos/$PROJECT_GH/releases/latest"
	if type "curl" >/dev/null; then
	DOWNLOAD_URL=$(curl -s $url \| grep $OS \| awk '/\"browser_download_url\":/{gsub( /[,\"]/,"", $2); print $2}')
	elif type "wget" >/dev/null; then
	DOWNLOAD_URL=$(wget -q -O - $url \| grep $OS \| awk '/\"browser_download_url\":/{gsub( /[,\"]/,"", $2); print $2}')
	fi
	fi

	version=$(git -C "$HELM_PLUGIN_DIR" describe --tags --exact-match 2>/dev/null \|\| :)
	if [ "$SCRIPT_MODE" = "install" -a -n "$version" ]; then
	DOWNLOAD_URL="/$PROJECT_GH/releases/download/$version/helm-diff-$OS-$ARCH.tgz"
	else
	DOWNLOAD_URL="/$PROJECT_GH/releases/latest/download/helm-diff-$OS-$ARCH.tgz"

Installation fails #322

Description

Activity

michaeljguarino commented on Jan 5, 2022

rkeidar commented on Jan 5, 2022

damienpontifex commented on Jan 5, 2022

damienpontifex commented on Jan 5, 2022

jBouyoud commented on Jan 5, 2022

mumoshu commented on Jan 5, 2022

damienpontifex commented on Jan 6, 2022

mumoshu commented on Jan 6, 2022

koenpunt commented on Jan 7, 2022

mumoshu commented on Jan 7, 2022

koenpunt commented on Jan 7, 2022

stitakis commented on Jan 7, 2022

14 remaining items

mumoshu commented on Jan 9, 2022

mumoshu commented on Jan 9, 2022

clemensutschig commented on Jan 9, 2022

mumoshu commented on Jan 9, 2022

mumoshu commented on Jan 9, 2022

mumoshu commented on Jan 9, 2022

--version 3.1.3

--version 3.2.0

without --version

michaelsauter commented on Jan 10, 2022

mumoshu commented on Jan 10, 2022

mumoshu commented on Jan 10, 2022

michaelsauter commented on Jan 10, 2022

mumoshu commented on Jan 10, 2022

Hammond95 commented on Jan 12, 2022

mumoshu commented on Jan 22, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

`--version 3.1.3`

`--version 3.2.0`

without `--version`