# US 2024/0005205 A1 · Iterative Attention-based Neural Network Training and Processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005205 A1
- Document ID: US20240005205A1
- Status: Published application
- Kind: A1
- Application number: 18/101,612
- Filed: 2023-01-26
- Published: 2024-01-04
- Earliest claimed: 2013-09-30
- Assignees: ManyWorlds, Inc.
- Inventors: Steven Dennis Flinn; Naomi Felina Moneypenny
- Web page: https://patentgrep.com/patent/US20240005205A1

## CPC

- G06N 20/00: Machine learning
- G06F 40/211: Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
- G06F 40/216: using statistical methods
- G06F 40/30: Semantic analysis
- G06N 3/045: Combinations of networks
- G06N 5/048: Fuzzy inferencing
- G06N 3/02: Neural networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An iterative attention-based neural network training and processing method and system iteratively applies a focus of attention of a trained neural network on syntactical elements and generates probabilities associated with representations of the syntactical elements, which in turn inform a subsequent focus of attention of the neural network, resulting in updated probabilities. The updated probabilities are then applied to generate syntactical elements for delivery to a user. The user may respond to the delivered syntactical elements, providing additional training information to the trained neural network.

## Claims

### Claim 1

A computer-implemented method comprising:
training a computer-implemented neural network;
accessing information comprising a first plurality of syntactical elements;
generating by application of the computer-implemented trained neural network a first plurality of probabilities that are associated with the first plurality of syntactical elements;
directing a first focus of attention of the computer-implemented trained neural network to a representation of a first subset of the first plurality of syntactical elements in accordance with the first plurality of probabilities;
generating a second plurality of probabilities in accordance with the first focus of attention of the computer-implemented trained neural network and the first plurality of probabilities;
directing a second focus of attention of the computer-implemented trained neural network to a representation of a second subset of the first plurality of syntactical elements in accordance with the second plurality of probabilities;
generating a third plurality of probabilities in accordance with the second focus of attention of the computer-implemented trained neural network and the second plurality of probabilities;
generating a second plurality of syntactical elements in accordance with the third plurality of probabilities; and
delivering the second plurality of syntactical elements to a user.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
training the computer-implemented neural network on a training set comprising a corpus of content comprising natural language.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
training the computer-implemented neural network on a training set comprising images associated with syntactical elements to infer syntactical elements corresponding to pixel patterns within images.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
accessing the information comprising the first plurality of syntactical elements, wherein the first plurality of syntactical elements comprises an interrogative communicated by the user.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
directing the first focus of attention of the computer-implemented trained neural network to the representation of a first subset of the first plurality of syntactical elements, wherein the representation comprises a vector.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
direct the second focus of attention in accordance with the second plurality of probabilities and a value of information.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
delivering the second plurality of syntactical elements to the user in response to a request by the user.

### Claim 8

A computer-implemented system comprising one or more processor-based devices configured to:
access information comprising a first plurality of syntactical elements;
generate by application of a computer-implemented trained neural network a first plurality of probabilities that are associated with the first plurality of syntactical elements;
direct a first focus of attention of the computer-implemented trained neural network to a representation of a first subset of the first plurality of syntactical elements in accordance with the first plurality of probabilities;
generate a second plurality of probabilities in accordance with the first focus of attention of the computer-implemented trained neural network and the first plurality of probabilities;
direct a second focus of attention of the computer-implemented trained neural network to a representation of a second subset of the first plurality of syntactical elements in accordance with the second plurality of probabilities;
generate a third plurality of probabilities in accordance with the second focus of attention of the computer-implemented trained neural network and the second plurality of probabilities;
generate a second plurality of syntactical elements in accordance with the third plurality of probabilities; and
deliver the second plurality of syntactical elements to a user.

### Claim 9 (depends on 8)

The system of claim 8, further comprising the one or more processor-based devices configured to:
access the information comprising a first plurality of syntactical elements, wherein the first plurality of syntactical elements comprises a request communicated by the user.

### Claim 10 (depends on 8)

The system of claim 8, further comprising the one or more processor-based devices configured to:
direct the first focus of attention of the computer-implemented trained neural network to the representation of a first subset of the first plurality of syntactical elements, wherein the representation comprises a vector.

### Claim 11 (depends on 8)

The system of claim 8, further comprising the one or more processor-based devices configured to:
direct the second focus of attention in accordance with the second plurality of probabilities and a value of information.

### Claim 12 (depends on 8)

The system of claim 8, further comprising the one or more processor-based devices configured to:
generate the second plurality of syntactical elements and at least one image in accordance with the third plurality of probabilities.

### Claim 13 (depends on 8)

The system of claim 8, further comprising the one or more processor-based devices configured to:
deliver the second plurality of syntactical elements to the user in response to a request by the user.

### Claim 14 (depends on 13)

The system of claim 13, further comprising the one or more processor-based devices configured to:
deliver the second plurality of syntactical elements to the user in response to the request by the user, wherein the user comprises a computer-implemented system.

### Claim 15

A computer-implemented method comprising:
accessing a first information comprising a first plurality of syntactical elements;
generating by application of a computer-implemented neural network a first plurality of probabilities that are associated with the first plurality of syntactical elements;
directing a first focus of attention of the computer-implemented neural network to a representation of a first subset of the first plurality of syntactical elements in accordance with the first plurality of probabilities;
generating a second plurality of probabilities in accordance with the first focus of attention of the computer-implemented neural network and the first plurality of probabilities;
directing a second focus of attention of the computer-implemented trained neural network to a representation of a second subset of the first plurality of syntactical elements in accordance with the second plurality of probabilities;
generating a third plurality of probabilities in accordance with the second focus of attention of the computer-implemented neural network and the second plurality of probabilities;
generating a second plurality of syntactical elements in accordance with the third plurality of probabilities;
receiving a second information in response to the generated second plurality of syntactical elements; and
generating a third plurality of syntactical elements in accordance with the second information.

### Claim 16 (depends on 15)

The method of claim 15, further comprising:
generating the second plurality of syntactical elements and an associated image.

### Claim 17 (depends on 15)

The method of claim 15, further comprising:
generating the second plurality of syntactical elements in accordance with the third plurality of probabilities and a value of information.

### Claim 18 (depends on 15)

The method of claim 15, further comprising:
generating the second plurality of syntactical elements in response to a request by a user.

### Claim 19 (depends on 15)

The method of claim 15, further comprising:
receiving the second information from a user to train the computer-implemented neural network.

### Claim 20 (depends on 15)

The method of claim 15, further comprising:
generating the third plurality of syntactical elements in accordance with the second information and the third plurality of probabilities.
