Skip to content

Workflow improvement suggestions #116

Open

Description

@rizar

I really like using smart_dispatch and I have a few ideas how it could be improved:

  • The location for creating logs should configurable, now SMART_DISPATCH_LOGS seems to be hardcoded.
  • The names of the log folder in SMART_DISPATCH_LOGS should be configurable. When I run my 99th experiment it becomes quite hard to enter the path to its folder with logs. Would be cool to be able to do smth like
smart_dispatch.py -n my_experiment_with_batch_norm

which could create a folder SMART_DISPATCH_LOGS/my_experiment_with_batch_norm

  • The by far most important deliverables of a launch are .err and .out files. I think they should have simpler names (now their names contain some kind of a hash) and that they should be in the top-level folder, not in /logs.
  • I think that the default current directory for the launched job should be not the home folder, but rather SMART_DISPATCH_LOGS/my_experiment_.... This would be very helpful when one wants to repeat an experiment multiple times but is too lazy to invent a handful of different names for experiments of the series.

So far this it it, maybe more is coming.

Activity

  1. mgermain commented on Apr 11, 2016

    @mgermain
    Member

    For the -n option #55 is related.

    " think that the default current directory for the launched job should be not the home folder,"
    It's not, It's the current directory.

  2. kudkudak commented on Nov 8, 2016

    @kudkudak

    👍 , also please allow for name with subdirectories, for instance I would run sth like

    smart_dispatch.py -n 2016_11_1/my_experiment_with_batch_norm
    

    , as for instance I like to group experiments by day or week

  3. mgermain commented on Feb 2, 2017

    @mgermain
    Member

    #132 is related to your point about the logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions