System, Method, and Computer Program Product for Saving Memory During Training of Knowledge Graph Neural Networks

A system, comprising:

• at least one processor configured to:
• receive a training dataset comprising a first set of knowledge graph embeddings associated with a plurality of entities for a first layer of a knowledge graph, wherein the plurality of entities comprises a plurality of users and a plurality of items;
• input the training dataset into a knowledge graph neural network;
• generate, based on inputting the training dataset into the knowledge graph neural network, at least one further set of knowledge graph embeddings associated with the plurality of entities for at least one further layer of the knowledge graph;
• quantize the at least one further set of knowledge graph embeddings to provide at least one set of quantized knowledge graph embeddings;
• store the at least one set of quantized knowledge graph embeddings in a memory without storing the at least one further set of knowledge graph embeddings;
• dequantize the at least one set of quantized knowledge graph embeddings to provide at least one set of dequantized knowledge graph embeddings, wherein the at least one set of dequantized knowledge graph embeddings approximates the at least one further set of knowledge graph embeddings;
• determine gradients for backpropagation based on the at least one set of dequantized knowledge graph embeddings;
• train parameters of the knowledge graph neural network based on the gradients;
• and
• determine a prediction indicating whether a user of the plurality of users will be interested in an item with which the user has not engaged based on the knowledge graph neural network.

A computer program product comprising at least one non-transitory computer-readable medium including program instructions that, when executed by at least one processor, cause the at least one processor to:

• receive a training dataset comprising a first set of knowledge graph embeddings associated with a plurality of entities for a first layer of a knowledge graph, wherein the plurality of entities comprises a plurality of users and a plurality of items;
• input the training dataset into a knowledge graph neural network;
• generate, based on inputting the training dataset into the knowledge graph neural network, at least one further set of knowledge graph embeddings associated with the plurality of entities for at least one further layer of the knowledge;
• quantize the at least one further set of knowledge graph embeddings to provide at least one set of quantized knowledge graph embeddings;
• store the at least one set of quantized knowledge graph embeddings in a memory without storing the at least one further set of knowledge graph embeddings;
• dequantize the at least one set of quantized knowledge graph embeddings to provide at least one set of dequantized knowledge graph embeddings, wherein the at least one set of dequantized knowledge graph embeddings approximates the at least one further set of knowledge graph embeddings;
• determine gradients for backpropagation based on the at least one set of dequantized knowledge graph embeddings;
• train parameters of the knowledge graph neural network based on the gradients;
• and
• determine a prediction indicating whether a user of the plurality of users will be interested in an item with which the user has not engaged based on the knowledge graph neural network.