US 11,769,035 B1 · Automatically determining configurations for executing recurrent neural networks

Abstract

Techniques are described automatically determining runtime configurations used to execute recurrent neural networks (RNNs) for training or inference. One such configuration involves determining whether to execute an RNN in a looped, or “rolled,” execution pattern or in a non-looped, or “unrolled,” execution pattern. Execution of an RNN using a rolled execution pattern generally consumes less memory resources than execution using an unrolled execution pattern, whereas execution of an RNN using an unrolled execution pattern typically executes faster. The configuration choice thus involves a time-memory tradeoff that can significantly affect the performance of the RNN execution. This determination is made automatically by a machine learning (ML) runtime by analyzing various factors such as, for example, a type of RNN being executed, the network structure of the RNN, characteristics of the input data to the RNN, an amount of computing resources available, and so forth.

Claims20

Dependent claims

A computer-implemented method performed by a machine learning (ML) runtime, the method comprising:

receiving a request to execute a recurrent neural network (RNN) using an input data sequence, the RNN defined by a network structure including a plurality of nodes organized into successive layers;
obtaining computing resource capacity information indicating an amount of computing resources available to the ML runtime at one or more computing devices at which the RNN is to be executed;
determining, using a separate ML model trained using RNN execution performance data in combination with information about whether the execution was performed using a rolled execution pattern or an unrolled execution pattern, and based on the network structure, the input data sequence, and the computing resource capacity information, an execution pattern for executing the RNN, the execution pattern selected from one of:
a rolled execution pattern or an unrolled execution pattern;
and
executing the RNN according to the execution pattern.

3 more elements

A system comprising:

a frontend implemented by a first one or more electronic devices of a provider network, the front end including instructions that upon execution cause the frontend to:
receive a request to execute a recurrent neural network (RNN) using an input data sequence, the RNN defined by a network structure including a plurality of nodes organized into successive layers, and
send the request to a machine learning (ML) service of the provider network;
and
the ML service implemented by a second one or more electronic devices, the ML service including instructions that upon execution cause the ML service to:
obtain computing resource capacity information indicating an amount of computing resources available to a ML runtime at one or more computing devices at which the RNN is to be executed;
determine, using a separate ML model trained using RNN execution performance data in combination with information about whether the execution was performed using a rolled execution pattern or an unrolled execution pattern, and based on the network structure, the input data sequence, and the computing resource capacity information, an execution pattern for executing the RNN, the execution pattern selected from one of:
a rolled execution pattern or an unrolled execution pattern;
and
execute the RNN according to the execution pattern.

7 more elements

Dependencies are inferred from the claim text.

Automatically determining configurations for executing recurrent neural networks

Life of this patent

Automatically determining configurations for executing recurrent neural networks

Life of this patent

Abstract

Claims20