Techniques for modifying and training a neural network
AssigneeNVIDIA Corporation
InventorsWentao Zhu, Daguang Xu, Can Zhao, Ziyue Xu, Holger Reinhard Roth
Life of this patent
Estimating the term…
Abstract
Apparatuses, systems, and techniques are described herein to speed up inferencing in a neural network by copying output from one layer of the neural network to another computing resource based on dependencies among layers in the network. In at least one embodiment, a processor comprising one or more circuits causes two or more subsequent layers of one or more neural networks to be performed on separate computing resources from a previous layer of the one or more neural networks.
Claims22
A processor, comprising:
- one or more circuits to cause two or more subsequent layers of one or more neural networks to be performed on separate computing resources from a precedent layer of the one or more neural networks based, at least in part, on whether a same output of the precedent layer is to be provided to both the two or more subsequent layers, wherein the same output of the precedent layer is to be provided to both the two or more subsequent layers by:
- determining that both the two or more subsequent layers are dependent on the same output from precedent layer;
- copying the same output of the precedent layer;
- and
- providing the same output of the precedent layer to a first subsequent layer of the two or more subsequent layers and a copied output to a second subsequent layer of the two or more subsequent layers.
A system, comprising:
- one or more computers having one or more processors to cause two or more subsequent layers of one or more neural networks to be performed on separate computing resources from a previous layer of the one or more neural networks based, at least in part, on whether a same output of the previous layer is to be provided to both the two or more subsequent layers, wherein the same output of the previous layer is to be provided to both the two or more subsequent layers by:
- determining that both the two or more subsequent layers are dependent on output from the previous layer;
- copying the output of the previous layer;
- and
- providing the output of the previous layer to a first subsequent layer of the two or more subsequent layers and a copied output to a second subsequent layer of the two or more subsequent layers.
A non-transitory machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to cause two or more subsequent layers of one or more neural networks to be performed on separate computing resources from a preceding layer of the one or more neural networks based, at least in part, on whether a same output of the preceding layer is to be provided to both the two or more subsequent layers, wherein the same output of the preceding layer is to be provided to both the two or more subsequent layers by:
- determining that both the two or more subsequent layers are dependent on output from preceding layer;
- copying the output of the preceding layer;
- and
- providing the output of the preceding layer to a first subsequent layer of the two or more subsequent layers and a copied output to a second subsequent layer of the two or more subsequent layers.
Dependencies are inferred from the claim text.