Iterative attention-based neural network training and processing
AssigneeSteven D. Flinn
InventorsSteven Dennis Flinn, Naomi Felina Moneypenny
Life of this patent
Estimating the term…
Abstract
An iterative attention-based neural network training and processing method and system iteratively applies a focus of attention of a trained neural network on syntactical elements and generates probabilities associated with representations of the syntactical elements, which in turn inform a subsequent focus of attention of the neural network, resulting in updated probabilities. The updated probabilities are then applied to generate syntactical elements for delivery to a user. The user may respond to the delivered syntactical elements, providing additional training information to the trained neural network.
Claims40
A system, comprising:
- one or more processors;
- one or more memories in communication with the one or more processors;
- and
- one or more programs, wherein the one or more programs are stored in the one or more memories and configured to be executed by the one or more processors, the one or more programs including instructions for causing:
- access to a first neural network that is trained by applying a first plurality of attentions to representations of different subsets of a first plurality of items of information;
- generation, by applying a second plurality of attentions utilizing the trained first neural network, of a second plurality of items of information;
- training of a second neural network by applying the second plurality of items of information;
- identification of a plurality of syntactical elements;
- generation, by applying a third plurality of attentions to representations of different subsets of the plurality of syntactical elements utilizing the trained second neural network, of a resulting item of content;
- and
- the resulting item of content to be sent to a user.
A computer-implemented method, comprising:
- at a system:
- causing access to a first neural network that is trained by applying a first plurality of attentions to representations of different subsets of first information, and that generates, by applying a second plurality of attentions and utilizing the trained first neural network, second information;
- causing training of a second neural network by applying the second information;
- causing identification of a plurality of items of content;
- causing generation, by applying a third plurality of attentions to representations of different subsets of the plurality of items of content and utilizing the trained second neural network, of a resulting item of content;
- and
- causing the resulting item of content to be sent to a user.
Dependencies are inferred from the claim text.