Parallelization plan generation for a neural network
AssigneeMicrosoft Technology Licensing, LLC
InventorsYoushan MIAO, Fan YANG, Quanlu ZHANG, Saeed MALEKI, Xu CAO, Yi ZHU, Mao YANG, Lidong ZHOU, Zhiqi LIN
Life of this patent
Estimating the term…
Abstract
Embodiments of the present disclosure include techniques for designing and generating a parallelization plan for a neural network so that workloads in the neural network may be split amongst multiple devices. Operators and tensors in the neural network are transformed into a set of functionally equivalent operators and tensors. These functionally equivalent operators and tensors are then scheduled to separate devices for execution.
Claims20
A system for generating a parallelization plan for a Neural Network (NN) model comprising:
- one or more processors;
- a non-transitory computer-readable medium storing a program executable by the one or more processors, the program comprising sets of instructions for:
- receiving a data flow graph representing the NN model, wherein the data flow graph includes a first operator having an input tensor and an output tensor;
- transforming the data flow graph, wherein transforming the data flow graph includes transforming the first operator into a set of operators that are functionally equivalent to the first operator;
- and
- assigning each operator in the set of operators to a computing device from a plurality of computing devices that are part of an execution environment configured to execute the NN model.
A method for generating a parallelization plan for a Neural Network (NN) model comprising:
- receiving a data flow graph representing the NN model, wherein the data flow graph includes a first operator having an input tensor and an output tensor;
- transforming the data flow graph, wherein transforming the data flow graph includes transforming the first operator into a set of operators that are functionally equivalent to the first operator;
- and
- assigning each operator in the set of operators to a computing device from a plurality of computing devices that are part of an execution environment configured to execute the NN model.
A non-transitory computer-readable medium storing a program executable by one or more processors, the program comprising sets of instructions for:
- receiving a data flow graph representing the NN model, wherein the data flow graph includes a first operator having an input tensor and an output tensor;
- transforming the data flow graph, wherein transforming the data flow graph includes transforming the first operator into a set of operators that are functionally equivalent to the first operator;
- and
- assigning each operator in the set of operators to a computing device from a plurality of computing devices that are part of an execution environment configured to execute the NN model.
Dependencies are inferred from the claim text.