Skip to content

分布式训练期间ps的cpu持续增长,训练效率下降问题 #1005

Description

@Christian9971

训练期间ps的cpu使用率变化情况

image

训练期间chief的cpu使用率变化情况(worker类似)

image

训练期间每秒训练批次变化情况

image

tf1.15版本,使用1chief 1ps 4worker进行分布式训练,训练期间ps的cpu持续增长,chief和worker的cpu后续有降低的情况,每秒训练批次也变少了,这是因为什么原因?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions