Memory management for machine learning training on gpu

A system for machine learning, the system comprising:

• a graphics processing unit, and
• a memory connected to the graphics processing unit,
• the graphics processing unit being configured to:
• invoke a no reuse detector;
• and
• launch a graphics processing unit kernel, after invoking the no reuse detector, the no reuse detector being configured to:
• identify a first data object having a persistence defined at least by one or more tasks for which the data object is an input;
• generate a dependency structure including:
• a first data object identifier identifying the first data object, and
• a first task, of the one or more tasks for which the data object is an input;
• determine a count, the count being a number of dependencies for the first data object identifier;
• determine that the first task completed execution;
• decrement the count by one based at least in part on determining that the first task completed execution;
• determine that the count is less than a first threshold;
• and
• deallocate the first data object based at least in part on determining that the count is less than the first threshold.

A method for machine learning, the method comprising:

• allocating memory for a first data object in a neural network;
• determining that the first data object has a persistence defined at least by one or more tasks for which the first data object is an input;
• determining that a last one of the one or more tasks for which the first data object is an input has completed execution;
• deallocating the first data object based on determining that the last one of the one or more tasks for which the first data object is an input has completed execution, and based on determining that the first data object has a persistence defined at least by one or more tasks for which the first data object is an input.