Repository navigation
Multivariate Polynomial Regression #239
Description
The tutorial is great, but here's where I'm at a loss: if I have multiple independent variables that I want to consider in the regression, how can I do that in a way that's similar to sklearns PolynomialFeatures: https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html#sklearn.preprocessing.PolynomialFeatures
I've been using MLJLinearModels.jl where applicable, but would it be better to just sub-in the SKLearnMLJ interface?
If there's a way to do it that easily builds off of the tutorial and I've just missed that, pointing me in the right direction is also appreciated.
Activity
ablaom commentedon Mar 1, 2025
Perhaps the InteractionTransformer is what you want. It takes a table of features and expands it to include monomials in the given features of some given order.
ablaom commentedon Mar 1, 2025
And you can use a pipeline to chain the transformer with your favourite linear model.
ablaom commentedon Mar 1, 2025
A non-MLJ alternative would be pre-process using an R-style formula, which is provided in Julia by StatsModels.jl. Unfortunately, there is no integration of that transformer with MLJ. I think you can use these formulas in GLM.jl, which is another provider of linear models.
MervynLarrierJr commentedon Mar 6, 2025
Thank you. I've used the interaction transformer, but since it doesn't provide the square terms I'd been trying to construct some transformation matrix based on some of the footballs i found to little success.
Out of curiosity, is there not a specified transformer because for univariate polynomial regression it's easy to create the polynomial? Or are polynomial models not really considered useful? I'm very new to machine learning so I'm just trying to build my intuition.
ablaom commentedon Mar 10, 2025
You are right!
InteractionTransformerincludes only monomials where degree of a given feature is 0 or 1. I had no idea. I've added an issue, as that seems unnecessarily limiting to me.Yeah, I guess most users are just creating these by hand (or use
@formulafrom StatsModels.jl).Certainly useful.