Reshape and broadcast optimizations to avoid unnecessary data movement

One or more non-transitory computer-readable storage media encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform actions comprising:

• detecting an original pattern of operations in a graph of computational operations on tensors, wherein the original pattern of operations is detected by identifying that an original reshape operation and at least one other operation in the graph are arranged according to a pre-defined pattern, wherein the original pattern of operations returns a final output tensor and takes as input an input tensor, wherein the original reshape operation in the original pattern (i) returns an original tensor and (ii) can be transformed to use less memory;
• transforming the original pattern of operations to a new pattern of operations with one or more reshape operations that return tensors smaller than the original tensor, wherein the new pattern of operations requires less data to be moved to satisfy alignment requirements for vector instructions or vector memory on a target hardware platform than the original pattern of operations;
• and
• generating executable code that is specific to the target hardware platform and that implements computations represented by the new pattern of operations.

A system comprising:

• one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform actions comprising:
• detecting an original pattern of operations in a graph of computational operations on tensors, wherein the original pattern of operations is detected by identifying that an original reshape operation and at least one other operation in the graph are arranged according to a pre-defined pattern, wherein the original pattern of operations returns a final output tensor and takes as input an input tensor, wherein the original reshape operation in the original pattern (i) returns an original tensor and (ii) can be transformed to use less memory;
• transforming the original pattern of operations to a new pattern of operations with one or more reshape operations that return tensors smaller than the original tensor, wherein the new pattern of operations requires less data to be moved to satisfy alignment requirements for vector instructions or vector memory on a target hardware platform than the original pattern of operations;
• and
• generating executable code that is specific to the target hardware platform and that implements computations represented by the new pattern of operations.