Is your feature request related to a problem or challenge? Please describe what you are trying to do.
We often need to create a DFField to represent the output of an Expr in a schema. We typically attempt to do this today based on an input schema.
Examples:
fn to_field(&self, input_schema: &DFSchema) in ExprSchemablepub fn exprlist_to_fields<'a>(expr: impl IntoIterator<Item = &'a Expr>, plan: &LogicalPlan) -> Result<Vec<DFField>>
This approach is problematic because the input schema loses a lot of information compared the plan it represents. For example, it might contain a column named substr(c0, 1, 2) and may not contain the column c0, making it impossible to reference c0 later on. This use case comes up during aggregates and is at least partly the cause of bugs such as #2430
Describe the solution you'd like
We should change the signatures of the functions above to accept an input plan instead of an input schema. This gives us more control over resolving expressions.
Describe alternatives you've considered
None
Additional context
None
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
We often need to create a
DFFieldto represent the output of anExprin a schema. We typically attempt to do this today based on an input schema.Examples:
fn to_field(&self, input_schema: &DFSchema)inExprSchemablepub fn exprlist_to_fields<'a>(expr: impl IntoIterator<Item = &'a Expr>, plan: &LogicalPlan) -> Result<Vec<DFField>>This approach is problematic because the input schema loses a lot of information compared the plan it represents. For example, it might contain a column named
substr(c0, 1, 2)and may not contain the columnc0, making it impossible to referencec0later on. This use case comes up during aggregates and is at least partly the cause of bugs such as #2430Describe the solution you'd like
We should change the signatures of the functions above to accept an input plan instead of an input schema. This gives us more control over resolving expressions.
Describe alternatives you've considered
None
Additional context
None