Skip to content

fix(publish): read the pypi trusted publishing audience from the index - #1374

Merged
BryanFRD merged 1 commit into
mainfrom
fix/pypi-oidc-audience
Nov 9, 2026
Merged

BryanFRD merged 1 commit into
mainfrom
fix/pypi-oidc-audience

Conversation

@BryanFRD

@BryanFRD BryanFRD commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Closes #1344

pypi_oidc::mint always asked GitHub for an OIDC token with audience=pypi, so trusted publishing to TestPyPI or another Warehouse index was refused at /_/oidc/mint-token.

  • GET <index>/_/oidc/audience and request the OIDC token for the returned audience (ureq query, so it is encoded and joins an existing query string).
  • If the lookup fails, pypi.org falls back to pypi; any other index errors naming the audience url, before GitHub is asked for a token.
  • Split mint_on(index, request_url, token) out of mint so the whole flow is tested against the local test server: a non-default audience (testpypi) end to end, an index without the endpoint, a refused exchange, a refused/garbled OIDC token.
  • Config docs updated to describe the audience lookup.

Warehouse instances do not share an OIDC audience (TestPyPI expects testpypi), so always requesting `pypi` made the mint-token exchange fail on any index but pypi.org, with an error pointing at the trusted publisher registration.

Query `<index>/_/oidc/audience` first, as pypa/gh-action-pypi-publish does, and request the GitHub OIDC token for that audience. Only pypi.org falls back to `pypi` when the lookup fails; any other index reports that it does not support trusted publishing.

Closes #1344

Signed-off-by: BryanFRD <bryanferrando59@gmail.com>
@BryanFRD
BryanFRD enabled auto-merge (squash) November 9, 2026 13:32

@ferrfleet-interne ferrfleet-interne Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve, with one security fix recommended before merge. Reading the audience from the index's /_/oidc/audience is correct (it fixes TestPyPI, whose audience is testpypi), and the new tests cover all three requests. Formal verification found a credential leak in an error message that this PR makes reachable on a new path.

To fix

  • low-medium, src/publishers/pypi_oidc.rs (index_base / index_audience): index_base keeps the whole authority of the registry url, so userinfo, ? and # carry into the index. With a registry url of https://ci:s3cr3t@127.0.0.1:<port>/simple/, a failed audience lookup returns E6102: publisher pypi: could not read the trusted publishing audience from https://ci:s3cr3t@127.0.0.1:<port>/_/oidc/audience; ..., so the credential is printed. Reproduced against the checked-out code. The mint-token error had the same leak before this PR, but now any non-pypi.org index whose audience lookup fails reaches it. Twine supports credentials in private index urls. Suggested fix: strip userinfo, query and fragment in index_base (or refuse them), and never put the raw url in the message.
  • minor, src/publishers/pypi_oidc.rs:100 (fallback_audience): it only matches the exact index https://pypi.org. A registry url of https://upload.pypi.org/legacy/ gives the index https://upload.pypi.org, so a failed lookup is a hard error instead of the pypi fallback. This contradicts the docs line saying pypi.org falls back. Either also match https://upload.pypi.org or narrow the docs wording. Formal verification notes the current behaviour matches "only pypi.org falls back" as documented, so this is a decision rather than a defect.

Formal verification

Bounded exhaustive tests (about 3.2M urls), not proofs.

Property Result
index_base accepts only https:// urls with a non-empty host, returns exactly https://<authority>, never panics passed
pypi audience fallback used only when the index is exactly https://pypi.org passed
Derived index is a bare origin (no userinfo, query, fragment) counterexample (see above)

One harness module (exhaustive, 3 tests) covers these properties; its code is below.

Harness code (appended to src/publishers/pypi_oidc.rs)
#[cfg(test)]
mod exhaustive {
    //! Bounded exhaustive checks, not proofs: every url built from one of
    //! `PREFIXES` and up to `MAX_SUFFIX` characters of `ALPHABET`.
    use super::*;

    const PREFIXES: [&str; 6] = ["https://", "http://", "", "HTTPS://", "https:/", "ftp://"];
    const ALPHABET: [char; 9] = ['a', '.', '/', ':', '?', '#', '@', 'é', '\\'];
    const MAX_SUFFIX: u32 = 6;

    fn each_url(mut check: impl FnMut(&str, &str, &str)) {
        for prefix in PREFIXES {
            for len in 0..=MAX_SUFFIX {
                for mut n in 0..ALPHABET.len().pow(len) {
                    let mut rest = String::new();
                    for _ in 0..len {
                        rest.push(ALPHABET[n % ALPHABET.len()]);
                        n /= ALPHABET.len();
                    }
                    check(prefix, &rest, &format!("{prefix}{rest}"));
                }
            }
        }
    }

    #[test]
    fn index_base_keeps_exactly_the_https_authority_and_refuses_the_rest() {
        each_url(|_, _, raw| {
            let authority = raw
                .strip_prefix("https://")
                .map(|rest| rest.split('/').next().unwrap());
            match index_base(Some(raw)) {
                Ok(index) => {
                    let authority = authority.expect("only https urls are accepted");
                    assert!(!authority.is_empty(), "{raw}");
                    assert_eq!(index, format!("https://{authority}"), "{raw}");
                    assert!(!index["https://".len()..].contains('/'), "{raw}");
                }
                Err(_) => assert!(authority.is_none_or(str::is_empty), "{raw}"),
            }
        });
    }

    #[test]
    fn index_base_is_a_bare_origin() {
        let mut leaks = Vec::new();
        each_url(|_, _, raw| {
            if let Ok(index) = index_base(Some(raw)) {
                if index["https://".len()..].contains(['?', '#', '@']) {
                    leaks.push((raw.to_string(), index));
                }
            }
        });
        assert!(leaks.is_empty(), "{} leaks, e.g. {:?}", leaks.len(), &leaks[..3.min(leaks.len())]);
    }

    #[test]
    fn only_pypi_org_falls_back_to_the_pypi_audience() {
        for raw in ["https://pypi.org", "https://pypi.org/", "https://pypi.org/legacy/"] {
            assert_eq!(fallback_audience(&index_base(Some(raw)).unwrap()), Some("pypi"));
        }
        each_url(|_, _, raw| {
            if let Ok(index) = index_base(Some(raw)) {
                assert_eq!(fallback_audience(&index).is_some(), index == "https://pypi.org");
            }
        });
        for raw in ["https://upload.pypi.org/legacy/", "https://PyPI.org/legacy/", "https://pypi.org:443/legacy/"] {
            assert_eq!(fallback_audience(&index_base(Some(raw)).unwrap()), None, "{raw}");
        }
    }
}

Not verified

  • Kani proofs of index_base: CBMC runs out of memory (8 GB) on anyhow::Error construction, so only the bounded tests above exist.
  • The mint_on / index_audience HTTP flow is covered by the PR's FakeServer tests (passing), not by verification.

@BryanFRD
BryanFRD merged commit 07919a6 into main Nov 9, 2026
35 checks passed
@BryanFRD
BryanFRD deleted the fix/pypi-oidc-audience branch November 9, 2026 14:18
ferrflow Bot added a commit that referenced this pull request Nov 9, 2026
## [7.35.4] - 2026-11-09

### Bug Fixes

- fix(publish): read the pypi trusted publishing audience from the index (#1374)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(publish): pypi trusted publishing always requests the pypi audience

1 participant