Skip to content

Prediction engine options - #5964

Merged
michaelgsharp merged 6 commits into
dotnet:mainfrom
michaelgsharp:predictionengine
Oct 12, 2021
Merged

michaelgsharp merged 6 commits into
dotnet:mainfrom
michaelgsharp:predictionengine

Conversation

@michaelgsharp

Copy link
Copy Markdown
Contributor

Fixes #5945

Currently when you dispose a prediction engine it also disposes the underlying model. This means that you are unable to cache a model to use multiple times if you want to create/dispose the prediction engine. This change makes it so the Prediction Engine no longer automatically disposes the model, allowing you to cache the model without worry about it being automatically disposed. You do now manually have to dispose of it.

@michaelgsharp
michaelgsharp requested a review from a team October 8, 2021 18:47
@michaelgsharp

Copy link
Copy Markdown
Contributor Author

@eerhardt I think this is a good change, but I would like your opinion as well. Besides the linked issue, there are also some email threads if you would like more information.

@sharwell

sharwell commented Oct 8, 2021

Copy link
Copy Markdown
Contributor

💡 The non-breaking way to approach this is to add a constructor (or similar) with a bool ownsTransformer parameter similar to XmlDictionaryWriter, or a bool leaveOpen parameter similar to BinaryReader.

There are examples of both directions of this pattern (true means keep open on dispose, or true means close on dispose). @stephentoub or @terrajobst are best positioned to state the current recommended semantics.

@michaelgsharp

Copy link
Copy Markdown
Contributor Author

Thats good to know. @stephentoub or @terrajobst do you have a preference?

@michaelgsharp

Copy link
Copy Markdown
Contributor Author

Looks like in our code base we have a couple of places where we are already doing it the same way as the XML. I'll convert it to that way, but if we want it the other way its fast to change.

@codecov

codecov Bot commented Oct 8, 2021 •

Copy link
Copy Markdown

Codecov Report

Merging #5964 (2189839) into main (54a3cb9) will decrease coverage by 0.05%.
The diff coverage is 22.72%.

@@            Coverage Diff             @@
##             main    #5964      +/-   ##
==========================================
- Coverage   68.29%   68.23%   -0.06%     
==========================================
  Files        1142     1142              
  Lines      242803   242821      +18     
  Branches    25385    25386       +1     
==========================================
- Hits       165825   165695     -130     
- Misses      70296    70433     +137     
- Partials     6682     6693      +11     
Flag Coverage Δ
Debug 68.23% <22.72%> (-0.06%) ⬇️
production 62.89% <22.72%> (-0.07%) ⬇️
test 88.61% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../Microsoft.ML.Data/Model/ModelOperationsCatalog.cs 53.60% <0.00%> (-1.78%) ⬇️
src/Microsoft.ML.TimeSeries/PredictionEngine.cs 76.96% <0.00%> (-3.63%) ⬇️
...c/Microsoft.ML.Data/Prediction/PredictionEngine.cs 87.83% <44.44%> (-4.81%) ⬇️
...rosoft.ML.Data/Model/PredictionEngineExtensions.cs 100.00% <100.00%> (ø)
...osoft.ML.KMeansClustering/KMeansPlusPlusTrainer.cs 83.60% <0.00%> (-7.16%) ⬇️
src/Microsoft.ML.Data/Training/TrainerUtils.cs 65.86% <0.00%> (-3.82%) ⬇️
...crosoft.ML.StandardTrainers/Standard/SdcaBinary.cs 85.06% <0.00%> (-3.08%) ⬇️
test/Microsoft.ML.AutoML.Tests/AutoFitTests.cs 87.07% <0.00%> (-1.97%) ⬇️
...crosoft.ML.StandardTrainers/Optimizer/Optimizer.cs 71.96% <0.00%> (-1.16%) ⬇️
...oft.ML.StandardTrainers/Standard/SdcaMulticlass.cs 91.12% <0.00%> (-1.03%) ⬇️
... and 6 more

@michaelgsharp michaelgsharp changed the title Prediction engine no longer disposes model. Prediction engine options Oct 9, 2021
{
/// <summary>
/// Options for the <see cref="PredictionEngine{TSrc, TDst}"/> as used in
/// [RandomizedPca(Options)](xref:Microsoft.ML.PcaCatalog.RandomizedPca(Microsoft.ML.AnomalyDetectionCatalog.AnomalyDetectionTrainers,Microsoft.ML.Trainers.RandomizedPcaTrainer.Options)).

@eerhardt eerhardt Oct 11, 2021 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a copy-paste error? #Resolved

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, resolved.

@eerhardt eerhardt left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>
@michaelgsharp
michaelgsharp merged commit 1dfccca into dotnet:main Oct 12, 2021
@michaelgsharp
michaelgsharp deleted the predictionengine branch October 12, 2021 01:19
@ghost ghost locked as resolved and limited conversation to collaborators Mar 17, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PredictionEngineBase Dispose() also calls ITransformer.Dispose()

3 participants