US 12,585,961 B1 · Data shuffle optimization

Abstract

An optimization algorithm is disclosed to reduce the number of data shuffle operators in a data flow graph representing a neural network model for a neural network. The optimization algorithm can identify single-entry single-exit (SESE) regions in the data flow graph and select the SESE regions that comprise only the data shuffle operators. An affine map for each selected SESE region can be generated from an input of the selected SESE region to an output of the selected SESE region. An affine shuffle operator corresponding to an affine map for a selected SESE region can replace that SESE region if an implementation cost of the affine map is lower than the implementation cost of the SESE region. Thus, by replacing the selected SESE regions comprising multiple data shuffle operators with corresponding affine shuffle operators, it is possible to reduce the total number of operators in the data flow graph and optimize the neural network model.

Claims20

Dependent claims

A computer-implemented method performed by a compiler to optimize a neural network model, comprising:

receiving a description of the neural network model;
generating an intermediate representation of the neural network model representing a data flow graph, the data flow graph comprising a plurality of operators;
identifying single-entry single-exit (SESE) regions in the data flow graph;
selecting one or more SESE regions in the identified SESE regions that do not include data computation operators;
for each region in the selected SESE regions, starting with an innermost region:
generating an affine map from an input of the region to an output of the region;
determining whether an implementation cost of the affine map is lower than an implementation cost of the region;
upon determining that the implementation cost of the affine map is lower than the implementation cost of the region, generating an affine shuffle operator that rearranges data elements at the input of the region into the output of the region based on the affine map;
and
replacing the region with the affine shuffle operator to reduce the data flow graph;
and
generating machine code that includes the affine shuffle operator instead of the replaced region for the neural network model from the reduced data flow graph.

9 more elements

A non-transitory computer readable medium having stored therein instructions that, when executed by one or more processors, cause the one or more processors to execute a compiler, the compiler performing operations including:

identifying single-entry single-exit (SESE) regions in a data flow graph for a neural network model, the data flow graph comprising a plurality of operators;
for each identified SESE region that does not include data computation operators:
generating an affine map from an input of the identified SESE region to an output of the identified SESE region;
generating an affine shuffle operator that rearranges data elements at the input of the identified SESE region into the output of the identified SESE region based on the affine map;
and
replacing the identified SESE region with the affine shuffle operator to reduce the data flow graph;
and
generating machine code that includes the affine shuffle operator instead of the replaced SESE region for the neural network model based on the reduced data flow graph.

5 more elements

Dependencies are inferred from the claim text.

Data shuffle optimization

Life of this patent

Data shuffle optimization

Life of this patent

Abstract

Claims20