Skip to content

Include the "save file" action (.ZIP file) as part of model.SaveFile() #1689

Closed

Description

@CESARDELATORRE

Currently, whenever we are saving a .ZIP model file, you always need to handle the code for the File stream class.

I think that is repetitive code that could be included in the API.
Instead of:

using (var fs = new FileStream(ModelPath, FileMode.Create, FileAccess.Write, FileShare.Write))
                mlContext.Model.Save(trainedModel, fs);

We could have an overridden method so I could simply do the following:

mlContext.Model.Save(trainedModel, modelPath);

Or even the following, if those methods were part of the model class itself instead of a utility class in the MLContext:

model.Save(modelPath);

Currently, this kind of line using the file stream class is something you need to repeat over and over in every/most training app (even when the constructor can be simplified):

using (var fs = new FileStream(ModelPath, FileMode.Create, FileAccess.Write, FileShare.Write))

Handling a FileStream object might should not be mandatory for the user but optional:

Of course, I would also maintain the current methods because in some cases you might want to just provide an existing stream object, but as opt-in.

Activity

  1. Zruty0 commented on Nov 21, 2018

    @Zruty0
    Contributor

    We already have model.SaveTo to save the model.
    @CESARDELATORRE , we wanted to follow up and see how the other .NET libraries go about IO: do they provide a pair of functionality for their IO, or only one?
    By pair I mean one method to IO from a stream, and another to IO from a string filePath.

  2. Zruty0 commented on Nov 21, 2018

    @Zruty0
    Contributor

    Also, there are shorter ways to create a file stream:

    using (var fs = File.Create(ModelPath))
  3. CESARDELATORRE commented on Nov 22, 2018

    @CESARDELATORRE
    ContributorAuthor

    Important to note is that, as I mentioned, it should also have the possibility of using an existing file stream.
    But in most of the cases you just want the simplest way which is to save the model's .ZIP file in a path, in a single line:

    model.Save(mlContext, filePath);

    For instance, in the case of the XMLDocument, it provides 4 overloads, by using stream, just the filepath, TextWriter and XMLWriter:

    https://docs.microsoft.com/en-us/dotnet/api/system.xml.xmldocument.save?view=netframework-4.7.2

    A few examples of .NET libraries where you just need to provide the file path name without having to deal with any stream:

    XML Document

    XmlDocument doc = new XmlDocument();
    doc.LoadXml("<item><name>wrench</name></item>");
    // change XML, etc...
    doc.Save("MyFile.xml");
    

    WORD Automation in C#:

    // Create new document.
    var doc = new DocumentModel();
    // Working with content...
    // Save as PDF file.
    doc.Save("Sample.pdf");
    

    VS Document.Save(path) Method

    https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2010/c0hhd7xt(v%3dvs.100)

    VS Graph Save(path) method

    https://docs.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2010/ff655497(v%3dvs.100)

    In Python / Scikit-Learn you also use a single API step: dump

    dump(model, 'filePathName')

    In contrast, we currently need to handle the FileStream object in between:

    This is what we have now:

    using (var fs = File.Create(ModelPath))
             trainedModel.SaveTo(mlContext, fs);
    

    or this

    using (var fs = new FileStream(ModelPath))
                mlContext.Model.Save(trainedModel, fs);
    

    But the simplest step would be to just provide the filePath:

    trainedModel.SaveTo(mlContext, filePath);

    Btw, I guess we always need to provide the mlContext in order to save the model sa a file, right?

    In any case, this is a minor improvement/simplication, not a big deal, but since the implementation is simple, that's why I'm suggesting it. :)

  4. JRAlexander commented on Nov 26, 2018

    @JRAlexander

    Can this also be made Async?

  5. glebuk commented on Jan 14, 2019

    @glebuk
    Contributor

    Please ensure that for model saving and loading there exist both stream and string based overloads.

  6. eerhardt commented on Mar 15, 2019

    @eerhardt
    Member

    Related #2983

  7. self-assigned this
    on Mar 15, 2019
  8. ghost locked as resolved and limited conversation to collaborators on Mar 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions