Methods and apparatus to facilitate tile-based gpu machine learning acceleration

An apparatus for machine learning processing, comprising:

• a memory;
• and
• at least one processor coupled to the memory and configured to:
• determine a tile size based on a memory size of a first memory and a job input size associated with executing a computational job, the computational job being one of a quantity of computational jobs configured to execute a machine learning primitive;
• load, based on the tile size, input data associated with a batch of computational jobs from a second memory to the first memory;
• generate batch output data by executing the batch of computational jobs using the input data loaded to the first memory;
• and
• store the generated batch output data to the second memory.

A non-transitory computer-readable medium storing computer executable code for machine learning processing, comprising code to:

• determine a tile size based on a memory size of a first memory and a job input size associated with executing a computational job, the computational job being one of a quantity of computational jobs configured to execute a machine learning primitive;
• load, based on the tile size, input data associated with a batch of computational jobs from a second memory to the first memory;
• generate batch output data by executing the batch of computational jobs using the input data loaded to the first memory;
• and
• store the generated batch output data to the second memory.