Skip to content

[Rust] [DataFusion] TableProvider should provide row count statistics #26722

Description

@asfimport

In order to start building a cost-based optimizer, we need some statistics about data sources. The most basic statistic would be number of rows.

I propose that we add a Statistics struct that initially just makes a total row count available but that we can later extend to support more advanced statistics.

struct Statistics {
  row_count: Option<usize>
} 

We can then add a method to TableProvider:

trait TableProvider {
  fn statistics() -> Option<Statistics>;
} 

Statistics should be optional because not all data sources can provide statistics.

Reporter: Andy Grove / @andygrove

Related issues:

PRs and other links:

Note: This issue was originally created as ARROW-10781. Please see the migration documentation for further details.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions