Back to results
US 12,754,769 B1Granted
Automated partitioning for one or more neural networks
AssigneeNVIDIA Corporation
InventorsSam Ginzburg, Sean Lee, Vinod Grover
Life of this patent
Estimating the term…
Abstract
Apparatuses, systems, and techniques are presented schedule operations to be performed. In at least one embodiment, two or more data partitions of a neural network are caused to be automatically scheduled to be performed by two or more corresponding logical processors in parallel.
Claims30
A processor, comprising:
- one or more circuits to:
- generate, as part of a first stage of a compiler, an intermediate representation of code to perform a neural network, the intermediate representation comprising one or more values indicating respective shape information of input values or output values of one or more intermediate computations;
- select two or more data partitions for the neural network to schedule performance by two or more processors in parallel, based, at least in part on the one or more values of the intermediate representation;
- and
- complete, as part of one or more second stages of the compiler, a schedule to perform the neural network by the two or more processors in parallel according to the selected two or more data partitions.
2 more elements
A system comprising:
- one or more processors to:
- generate, as part of a first stage of a compiler, an intermediate representation of code to perform a neural network, the intermediate representation comprising one or more values indicating respective shape information of input values or output values of one or more intermediate computations;
- select two or more data partitions for the neural network to schedule performance of the neural network by two or more processors in parallel, based, at least in part on the one or more values of the intermediate representation;
- and
- complete, as part of one or more second stages of the compiler, a schedule to perform the neural network by the two or more processors in parallel according to the selected two or more data partitions.
2 more elements
A method comprising:
- generating, as part of a first stage of a compiler, an intermediate representation of code to perform a neural network, the intermediate representation comprising one or more values indicating respective shape information of input values or output values of one or more intermediate computations;
- selecting two or more data partitions for the neural network to schedule performance of the neural network by two or more processors in parallel, based, at least in part on the one or more values of the intermediate representation;
- and
- completing, as part of one or more second stages of the compiler, a schedule to perform the neural network by the two or more processors in parallel according to the selected two or more data partitions.
1 more element
A non-transitory computer machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:
- generate, as part of a first stage of a compiler, an intermediate representation of code to perform a neural network, the intermediate representation comprising one or more values indicating respective shape information of input values or output values of one or more intermediate computations;
- select two or more data partitions for the neural network to schedule performance of the neural network by two or more processors in parallel, based, at least in part on the one or more values of the intermediate representation;
- and
- complete, as part of one or more second stages of the compiler, a schedule to perform the neural network by the two or more processors in parallel according to the selected two or more data partitions.
1 more element
An inferencing system, comprising:
- one or more processors to:
- generate, as part of a first stage of a compiler, an intermediate representation of code to perform a neural network, the intermediate representation comprising one or more values indicating respective shape information of input values or output values of one or more intermediate computations;
- select two or more data partitions for the neural network to schedule performance of the neural network by two or more processors in parallel, based, at least in part on the one or more values of the intermediate representation;
- and
- complete, as part of one or more second stages of the compiler, a schedule to perform the neural network by the two or more processors in parallel according to the selected two or more data partitions;
- and
- memory for storing network parameters for the neural network.
4 more elements
Dependencies are inferred from the claim text.