GitLab Token (env) not found after update

[weberpatr]

Hello,

I just updated my SourceBot instance from v4.6.7 to latest 4.10.3.
I also applied the required configuration change about the required ENV CONFIG_PATH.
But now I struggle about the token configuration.

config.json:

{
    "$schema": "https://raw.githubusercontent.com/sourcebot-dev/sourcebot/main/schemas/v3/index.json",
    "connections": {
            "company-gitlab": {
                  "all": true,
                  "url": "https://gitlab.company.io",
                  "type": "gitlab",
                  "token": {
                    "env": "GITLAB_TOKEN"
                  },
                  "exclude": {
                    "archived": true,
                    "userOwnedProjects": true
                  }
            }
    }
}

compose.yaml:

  sourcebot:
    container_name: sourcebot
    image: ghcr.io/sourcebot-dev/sourcebot:v4.10.3
    restart: unless-stopped
    environment:
      CONFIG_PATH: "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/data/config.json"
      GITLAB_TOKEN: "glpat-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
      AUTH_URL: "https://sourcebot.company.io"
      SOURCEBOT_TELEMETRY_DISABLED: "true"
    volumes:
      - /opt/sourcebot/data:/data
    ports:
      - "3000:3000"

ENV command inside docker container

# env | grep GITLAB
GITLAB_TOKEN=glpat-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

But the UI reports the connection as failed with the message: Environment variable GITLAB_TOKEN not found..
I also tried adding the secret as env to data/.sourcebot/.secret with the same result.

Any idea what I'm missing?

[brendan-kellam]

I asked the Sourcebot agent to try and debug this, and it came up with what you see below. I think it would be most useful if you could try troubleshooting step #2 below to see if the environment variable is available to the node process.

---

Based on the analysis of the Sourcebot codebase and the issue description, here is the explanation and suggested solution.

Analysis

The error Environment variable GITLAB_TOKEN not found. is thrown by the getTokenFromConfig function in packages/shared/src/crypto.ts crypto.ts:81-86.

export const getTokenFromConfig = async (token: Token): Promise<string> => {
    if ('env' in token) {
        const envToken = process.env[token.env];
        if (!envToken) {
            throw new Error(`Environment variable ${token.env} not found.`);
        }
        return envToken;
    }
    // ...
};

This function directly accesses process.env using the key specified in your config.json (GITLAB_TOKEN). Since you verified that the environment variable exists inside the container using env | grep GITLAB, the issue is likely that the Node.js process running the application is not seeing this variable.
This can happen for a few reasons in a containerized environment managed by supervisord (which Sourcebot uses):

1. Process Environment Isolation: Although supervisord usually inherits environment variables, the specific way the application is started might be stripping them if they are not explicitly passed or if the shell context is different.
2. Config Misconfiguration: A subtle typo or hidden character in config.json (e.g., "env": "GITLAB_TOKEN " with a space) would cause process.env lookup to fail.
3. Web vs Backend: The error might be coming from the Web process (Next.js) rather than the Backend process. The Web process also validates connections and needs access to the same environment variables.

Troubleshooting Steps

1. Check for Hidden Characters in config.json

Ensure there are no trailing spaces or hidden characters in your config.json.

"token": {
  "env": "GITLAB_TOKEN"
}

If you copied this from a source that had non-breaking spaces or invisible characters, the key lookup process.env["GITLAB_TOKEN "] will fail even if GITLAB_TOKEN exists.

2. Verify Variable Availability to the Node Process

The env command in the shell shows variables for that shell, but the node process runs as a child of supervisord. To verify what the node process actually sees, you can inspect its environment:

1. Exec into the container: docker exec -it sourcebot /bin/sh

2. Find the PID of the backend or web process: pgrep -f "node packages/backend" or pgrep -f "node packages/web"

3. Check the environment of that PID:

   cat /proc/<PID>/environ | tr '\0' '\n' | grep GITLAB

   If this returns nothing, the variable is not reaching the application process.

3. Workaround: Use .sourcebot/.secret (Correctly)

You mentioned trying to add the secret to data/.sourcebot/.secret. This file is sourced by entrypoint.sh entrypoint.sh:118, but only if SOURCEBOT_ENCRYPTION_KEY is NOT already set in the environment entrypoint.sh:106.

If SOURCEBOT_ENCRYPTION_KEY is generated or passed via env, the .secret file might be skipped or not fully sourced depending on the logic flow.

Recommended Fix:

Ensure GITLAB_TOKEN is defined in your compose.yaml (as you have done) and try restarting the container with docker compose down && docker compose up -d to ensure a fresh environment, as docker compose restart sometimes preserves old contexts.

4. Alternative: Use environmentOverrides

If the environment variable propagation continues to fail, you can try defining the token directly in config.json using environmentOverrides (though less secure for secrets, it verifies if the app can read the config):

{
    "environmentOverrides": {
        "GITLAB_TOKEN": {
            "type": "string",
            "value": "glpat-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
        }
    },
    "connections": { ... }
}

This forces the application to populate GITLAB_TOKEN from the config file logic in env.server.ts env.server.ts:16-46.

Summary

The code confirms that GITLAB_TOKEN must be present in process.env. The error is legitimate: the Node.js process cannot find it. The most likely cause is a subtle configuration issue (whitespace in JSON) or Docker environment propagation issue which a full down/up cycle often resolves.

[weberpatr]

Hello,

Thanks for the feedback.
I can confirm GITLAB_TOKEN is also available in the Node process.

But what I just noticed:
Sourcebot was still syncing the repositories without any issues, although the Connection page indicated a failed status.
Now the status changed to Completed without any change from my side.
The error Environment variable GITLAB_TOKEN not found. was only shown in the UI, when I hovered on the failed icon - the error did not appear in the logfile.

Therefore I guess the problem has solved itself somehow :)