Loading patent…
AssigneeSambaNova Systems, Inc.
InventorsMingran WANG, Leon ZHANG
Estimating the term…
Systems, methods, and computer program products for computing a gradient on a reconfigurable dataflow processor are disclosed. A host computer receives a compute graph representing a gradient computation on a first tensor and a second tensor that share a batch dimension of length B. The host computer transforms the compute graph by replacing a gradient node with B parallel matrix multiplication nodes, each of which operates on a respective batch slice of the first tensor and a respective batch slice of the second tensor. Outputs of the B parallel matrix multiplication nodes are routed to at least one addition node that sums the outputs to produce a gradient value. A reconfigurable dataflow processor executes the transformed compute graph. The transformation reduces on-chip memory consumption, alleviates place and route congestion, and improves throughput for gradient calculations used in neural network training.
A system for computing a gradient on a reconfigurable dataflow processor, the system comprising:
A method for computing a gradient on a reconfigurable dataflow processor, the method comprising:
A non-transitory computer readable storage medium having program instructions embodied therewith, wherein the program instructions are executable by a processor to cause the processor to perform operations comprising:
Dependencies are inferred from the claim text.