# US 2023/0058522 A1 · Adaptive feedback timing system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0058522 A1
- Document ID: US20230058522A1
- Status: Published application
- Kind: A1
- Application number: 17/404,304
- Filed: 2021-08-17
- Published: 2023-02-23
- Earliest claimed: 2021-08-17
- Assignees: The Boeing Company
- Applicants: The Boeing Company
- Inventors: Cristina Boyer
- Web page: https://patentgrep.com/patent/US20230058522A1

## CPC

- G06F 11/3438: {monitoring of user actions (tracking the activity of the user H04L67/535)}
- G09B 7/04: characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question or supplying a further explanation
- G06F 11/3457: {Performance evaluation by simulation}
- G09B 5/065: {Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems}
- G09B 7/00: Electrically-operated teaching apparatus or devices working with questions and answers (mechanically operated G09B3/00; computing arrangements G06F)
- G09B 7/08: characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question or supplying further information
- G09B 9/301: {by computer-processed or -generated image}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

An adaptive feedback timing system and method includes receiving, by a performance observation system, monitoring data associated with electronically monitoring a lesson by a variable feedback teaching device. Adaptive feedback timing also includes receiving, by the performance observation system, error detection data associated with the variable feedback teaching device automatically detecting an error made by a student during the lesson. After receiving the error detection data, a feedback pattern is automatically selected based on a performance history criterion. Feedback data is then communicated to the variable feedback teaching device for presentation to the student according to the automatically selected feedback pattern.

## Claims

### Claim 1

A method comprising:
receiving, by a performance observation system, monitoring data associated with electronically monitoring a lesson by a variable feedback teaching device;
receiving, by the performance observation system, error detection data associated with the variable feedback teaching device automatically detecting an error made by a student during the lesson;
after receiving the error detection data, automatically selecting a feedback pattern based on a performance history criterion; and
communicating feedback data to the variable feedback teaching device for presentation to the student according to the automatically selected feedback pattern.

### Claim 2 (depends on 1)

The method of claim 1, wherein the feedback pattern is automatically selected from among an immediate feedback pattern, a breakpoint feedback pattern, and an end-of-lesson feedback pattern.

### Claim 3 (depends on 1)

The method of claim 1, wherein the performance history criterion comprises a past performance metric associated with the student at a point in the lesson at which the error was automatically detected.

### Claim 4 (depends on 3)

The method of claim 3, wherein the past performance metric indicates whether the student has previously committed the error, and wherein automatically selecting the feedback pattern comprises automatically selecting, from a plurality of feedback patterns, a particular feedback pattern that causes feedback to be provided to the student according to a particular timing based on whether the student has previously committed the error.

### Claim 5 (depends on 1)

The method of claim 1, wherein the performance history criterion comprises a past performance metric associated with the student for the lesson.

### Claim 6 (depends on 5)

The method of claim 5, wherein the past performance metric indicates whether the student has previously performed below a success threshold for the lesson, and wherein automatically selecting the feedback pattern comprises automatically selecting, from a plurality of feedback patterns, a particular feedback pattern that causes feedback to be provided to the student according to a particular timing based on whether the student has previously performed below the success threshold.

### Claim 7 (depends on 1)

The method of claim 1, wherein the performance history criterion comprises a past performance metric associated with the student for a second lesson.

### Claim 8 (depends on 7)

The method of claim 7, wherein the past performance metric indicates whether the student has made a second error in the second lesson, wherein the second error is thematically similar to the error, and wherein automatically selecting the feedback pattern comprises, in response to determining that the error is thematically similar to the second error, automatically selecting, from a plurality of feedback patterns, a particular feedback pattern that causes feedback to be provided to the student according to a particular timing based on whether the student committed the second error.

### Claim 9 (depends on 8)

The method of claim 8, further comprising determining that the error is thematically similar to the second error based on the error occurring during a portion of the lesson having the same subject matter as a portion of the second lesson during which the student made the second error.

### Claim 10 (depends on 1)

The method of claim 1, wherein the performance history criterion comprises a past performance metric associated with a plurality of students at a point in the lesson at which the error was automatically detected.

### Claim 11 (depends on 10)

The method of claim 10, wherein the past performance metric indicates a count of additional students that have committed the error, and wherein automatically selecting the feedback pattern comprises automatically selecting, from a plurality of feedback patterns, a particular feedback pattern that causes feedback to be provided to the student according to a particular timing based on a comparison of the count of additional students that committed the error to a group performance threshold.

### Claim 12 (depends on 1)

The method of claim 1, wherein the performance history criterion comprises a past performance metric associated with a plurality of students for the lesson or with a plurality of students for a second lesson.

### Claim 13 (depends on 1)

The method of claim 1, further comprising, after automatically detecting the error, classifying the error according to an error category, wherein the error category is one of an incorrect answer category, an out of sequence category, or a wrong state category.

### Claim 14 (depends on 13)

The method of claim 13, wherein the performance history criterion comprises a past performance metric associated with a student’s performance in the error category.

### Claim 15 (depends on 14)

The method of claim 14, wherein the past performance metric indicates whether the student has previously performed below a success threshold associated with the error category, and wherein automatically selecting the feedback pattern comprises automatically selecting, from a plurality of feedback patterns, a particular feedback pattern that causes feedback to be provided to the student according to a particular timing based on whether the student has previously performed below the success threshold associated with the error category.

### Claim 16

A non-transient, computer-readable medium storing instructions executable by one or more processors to perform operations comprising:
receiving, by a performance observation system, monitoring data associated with electronically monitoring a lesson by a variable feedback teaching device;
receiving, by the performance observation system, error detection data associated with the variable feedback teaching device automatically detecting an error made by a student during the lesson;
after receiving the error detection data, automatically selecting a feedback pattern based on a performance history criterion; and
communicating feedback data to the variable feedback teaching device for presentation to the student according to the automatically selected feedback pattern.

### Claim 17 (depends on 17)

The non-transient, computer-readable medium of claim 17, wherein the feedback pattern is automatically selected from among an immediate feedback pattern, a breakpoint feedback pattern, and an end-of-lesson feedback pattern.

### Claim 18

A system comprising:
a memory configured to store instructions; and
one or more processors configured to:
receive, by a performance observation system, monitoring data associated with electronically monitoring a lesson by a variable feedback teaching device;
receive, by the performance observation system, error detection data associated with the variable feedback teaching device automatically detecting an error made by a student during the lesson;
after receiving the error detection data, automatically select a feedback pattern based on a performance history criterion; and
communicate feedback data to the variable feedback teaching device for presentation to the student according to the automatically selected feedback pattern.

### Claim 19 (depends on 19)

The system of claim 19, wherein the variable feedback teaching device is incorporated into a cockpit simulator.

### Claim 20 (depends on 19)

The system of claim 19, wherein the variable feedback teaching device is coupled to a cockpit simulator.
