Dynamic minibatch sizes

A method for processing an embedding layer of a neural network implemented on an integrated circuit, the method comprising:

• receiving, by a first host, an instruction to execute a batch of input features;
• dividing, by the first host, the batch of input features into N minibatches;
• for each of the N minibatches, obtaining, for each input feature in the minibatch, a corresponding embedding to the input feature from a trainable lookup table;
• processing, by the first host, the corresponding embeddings that were obtained from the trainable lookup table for each input feature in each of the N minibatches;
• and
• generating, for each input feature in the batch of input features, an embedding layer output in response to processing the corresponding embedding obtained for the input feature.

A system for processing an embedding layer of a neural network implemented on an integrated circuit, the system comprising:

• a processing device;
• and
• a non-transitory machine-readable storage device storing instructions that are executable by the processing device to cause performance of operations comprising:
• receiving, by a first host, an instruction to execute a batch of input features;
• dividing, by the first host, the batch of input features into N minibatches;
• for each of the N minibatches, obtaining, for each input feature in the minibatch, a corresponding embedding to the input feature from a trainable lookup table;
• processing, by the first host, the corresponding embeddings that were obtained from the trainable lookup table for each input feature in each of the N minibatches;
• and
• generating, for each input feature in the batch of input features, an embedding layer output in response to processing the corresponding embedding obtained for the input feature.

A non-transitory machine-readable storage device storing instructions for processing an embedding layer of a neural network implemented on an integrated circuit, the instructions being executable by a processing device to cause performance of operations comprising:

• receiving, by a first host, an instruction to execute a batch of input features;
• dividing, by the first host, the batch of input features into N minibatches;
• for each of the N minibatches, obtaining, for each input feature in the minibatch, a corresponding embedding to the input feature from a trainable lookup table;
• processing, by the first host, the corresponding embeddings that were obtained from the trainable lookup table for each input feature in each of the N minibatches;
• and
• generating, for each input feature in the batch of input features, an embedding layer output in response to processing the corresponding embedding obtained for the input feature.