Skip to content

Upgrade rand to 0.10, rand_distr to 0.6 #21839

Open
Feature
Open
Upgrade rand to 0.10, rand_distr to 0.6#21839
Feature
Labels
enhancementNew feature or request

Description

@Jefffrey
Jefffrey
opened on Apr 25, 2026

Is your feature request related to a problem or challenge?

Dependabot has a PR to upgrade rand_distr:

This is currently blocked as it requires rand = "0.10":

However it seems dependabot isn't raising a PR for rand = "0.10" because it removed a feature we use, small_rng.

Describe the solution you'd like

Manually upgrade to rand = "0.10" and rand_dist = "0.6"

datafusion/datafusion/core/Cargo.toml

Lines 175 to 176 in 89e14f1

rand = { workspace = true, features = ["small_rng"] }
rand_distr = "0.5"

datafusion/Cargo.toml

Line 188 in 89e14f1

rand = "0.9"

What is involved:

datafusion/datafusion/common/benches/with_hashes.rs

Line 238 in 89e14f1

rng.clone().sample_iter(&Alphanumeric).take(len).collect();

datafusion/datafusion/functions/src/math/random.rs

Line 85 in 89e14f1

rng.fill(&mut values[..]);

  • arrow-rs still depends on rand = "0.9" and we interface with this, so we need to upgrade arrow-rs
    • create_primitive_array comes from arrow-rs

datafusion/datafusion/functions-aggregate/benches/array_agg.rs

Line 91 in 89e14f1

let values = create_primitive_array::<T>(length, 0.0);

Describe alternatives you've considered

No response

Additional context

See changelog for rand:

Activity

  1. Jefffrey
    added the
    Feature
    issue type on Apr 25, 2026
  2. Jefffrey
    added
    enhancementNew feature or request
    on Apr 25, 2026

  3. getChan commented on Apr 26, 2026

    @getChan
    getChan
    on Apr 26, 2026
    Contributor

    I took a pass at this and it looks like the remaining rough edge is arrow-rs still depending on rand 0.9 in places we interact with (for example some bench/test utilities). We can work around that locally, but that feels like the wrong direction.

    I think this will be much cleaner once the relevant arrow-rs pieces are updated to rand 0.10 as well. My preference would be to defer this until then rather than add local compatibility shims/workarounds on the DataFusion side.

  4. Jefffrey commented on Apr 26, 2026

    @Jefffrey
    Jefffrey
    on Apr 26, 2026
    ContributorAuthor

    It might actually be better to pin those places that depends on arrow-rs to 0.9 while we upgrade the other places, if possible. Otherwise we'd have to do these changes alongside an arrow-rs version bump.

  5. comphead commented on Apr 27, 2026

    @comphead
    comphead
    on Apr 27, 2026
    Contributor

    Hope this would be easier than upgrade to 0.9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    Feature

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions