Lossless tiling in convolution networks—read-modify-write in backward pass

A computer implemented method comprising:

• writing, in a memory and in an overlapping configuration, a plurality of output tiles of an output tensor generated by a first subgraph of a processing graph, wherein an overlapping region between any two neighboring output tiles of the plurality of output tiles comprises an aggregate of at least (i) a corresponding region of a first neighboring output tile and cii) a corresponding region of a second neighboring output tile, such that aggregate data in a segment of the overlapping region is an aggregate of at least (A) first data in a corresponding segment of the corresponding region of the first neighboring output tile and (B) second data in a corresponding segment of the corresponding region of the second neighboring output tile;
• tiling at least a section of the output tensor to generate a plurality of non-overlapping input tiles of an input tensor;
• processing the plurality of non-overlapping input tiles of the input tensor by a second subgraph of the processing graph;
• and
• training weights of a neural network model, based at least in part on an output of the second subgraph of the processing graph.

A non-transitory computer readable storage medium impressed with computer program instructions, the instructions, when executed on a processor, implement a method comprising:

• generating, by a first subgraph of a processing graph, a plurality of overlapping output tiles of an output tensor;
• dividing the output tensor in (i) a first section and (ii) a second section that fully surrounds the first section;
• tiling the first section of the output tensor to generate a plurality of non-overlapping input tiles of an input tensor;
• refraining from tiling the second section of the output tensor, such that the second section of the output tensor is not included in the plurality of non-overlapping input tiles of the input tensor,
• wherein the input tensor includes (i) the first section that is tiled in the plurality of non-overlapping input tiles and (ii) the second section that is not tiled and that fully surrounds the first section;
• processing the plurality of non-overlapping input tiles of the input tensor by a second subgraph of the processing graph;
• and
• training weights of a neural network model, based at least in part on an output of the second subgraph of the processing graph.

A computer implemented method comprising:

• initializing an area of a memory to first data that comprises all zeros;
• generating a first output tile;
• reading the first data comprising all zeros from the area of the memory, adding the first output tile to a first section of the first data to generate second data, and writing the second data to the area of the memory;
• generating a second output tile;
• reading the second data from the area of the memory, adding the second output tile to a second section of the second data to generate third data, and writing the third data to the memory;
• and
• training weights of a neural network model, based at least in part on the third data.