# US 12,283,198 B2 · Systems and methods for automatically revising feedback in electronic learning systems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,283,198 B2
- Document ID: US12283198B2
- Status: Granted
- Kind: B2
- Application number: 17/719,665
- Filed: 2022-04-13
- Published: 2022-10-13
- Granted: 2025-04-22
- Earliest claimed: 2021-04-13
- Assignees: D2L Corporation
- Applicants: D2L Corporation
- Inventors: Brian Cepuran; Jeremy Auger; John Baker
- Web page: https://patentgrep.com/patent/US12283198B2

## CPC

- G09B 7/02: of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student
- G06F 40/166: Editing, e.g. inserting or deleting
- G06F 40/253: Grammatical analysis; Style critique
- G06F 40/279: Recognition of textual entities
- G06F 40/30: Semantic analysis
- G06N 20/00: Machine learning
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods for automatically revising feedback in an electronic learning system are provided. The method involves operating at least one processor to: receive feedback text data submitted by a user to evaluate another user; identify a plurality of sentiment groups in the feedback text data, each sentiment group consisting of a portion of the feedback text data associated with a common sentiment; select at least one feedback processing module to process each sentiment group, the at least one feedback processing module comprising at least one machine-learned model; for each sentiment group, process the corresponding portion of the feedback text data using the at least one machine-learned model to determine at least one suggested revision for the portion of the feedback text data; and generate a revised version of the feedback text data indicating each suggested revision for the feedback text data.

## Claims

### Claim 1

A method for automatically revising feedback in an electronic learning system, the method comprising operating at least one processor to:
receive, from a computing device associated with a reviewer, initial feedback text data submitted by the reviewer to evaluate electronic input submitted by a learner, the reviewer and the learner being users of the electronic learning system, the electronic input comprising an evaluative submission;
divide the initial feedback text data submitted by the reviewer into a plurality of portions;
detect a sentiment associated with each portion of the initial feedback text data submitted by the reviewer;
identify a plurality of sentiment groups in the initial feedback text data submitted by the reviewer, each sentiment group consisting of one or more portions of the initial feedback text data associated with a common sentiment;
select at least one feedback processing module based on the one or more portions of the initial feedback text data corresponding to the sentiment group, the at least one feedback processing module comprising at least one machine-learned model;
for each sentiment group, apply the at least one machine-learned model of the selected at least one feedback processing module to the corresponding one or more portions of the initial feedback text data submitted by the reviewer to determine at least one suggested revision for the portion of the initial feedback text data submitted by the reviewer; and
generate revised feedback text data indicating each suggested revision for the initial feedback text data submitted by the reviewer.

### Claim 2 (depends on 1)

The method of claim 1, wherein operating the at least one processor to identify the plurality of sentiment groups comprises operating the at least one processor to identify a plurality of emotionally charged words in the initial feedback text data submitted by the reviewer, and wherein at least one sentiment group includes a portion of the initial feedback text data corresponding to at least one of the emotionally charged words.

### Claim 3 (depends on 1)

The method of claim 1 comprises operating the at least one processor to, for each sentiment group:
determine an emotional polarity associated with the corresponding one or more portions of the initial feedback text data submitted by the reviewer; and
determine the at least one suggested revision based on the associated emotion.

### Claim 4 (depends on 3)

The method of claim 3, wherein operating the at least one processor to determine the emotional polarity associated with the corresponding one or more portions of the initial feedback text data submitted by the reviewer comprises operating the at least one processor to:
identify, in the corresponding one or more portions of the initial feedback text data submitted by the reviewer, at least one of: capitalization, punctuation, or emotionally charged words.

### Claim 5 (depends on 1)

The method of claim 1 comprises operating the at least one processor to, for each sentiment group:
determine a specificity of the corresponding one or more portions of the initial feedback text data submitted by reviewer; and
determine the at least one suggested revision based on the specificity.

### Claim 6 (depends on 1)

The method of claim 1, further comprising operating the at least one processor to determine an effectiveness of the revised feedback text data.

### Claim 7 (depends on 1)

The method of claim 1 comprises operating the at least one processor to:
retrieve previous feedback text data for the learner based on an effectiveness of the previous feedback text data; and
determine the at least one suggested revision is based on the previous feedback text data.

### Claim 8 (depends on 1)

The method of claim 1 comprises operating the at least one processor to:
retrieve previous feedback text data submitted by the reviewer based on an effectiveness of the previous feedback text data; and
determine the at least one suggested revision is based on the previous feedback text data.

### Claim 9 (depends on 1)

The method of claim 1 comprises operating the at least one processor to:
determine the at least one suggested revision based on the evaluative submission.

### Claim 10 (depends on 1)

The method of claim 1, wherein operating the at least one processor to receive the initial feedback text data submitted by the reviewer comprises receiving initial feedback audio data and convert the initial feedback audio data to the initial feedback text data.

### Claim 11

A system for automatically revising feedback in an electronic learning system, the system comprising:
at least one processor coupled to a communication interface, the at least one processor operable to:
receive, from a computing device associated with a reviewer, initial feedback text data submitted by the reviewer to evaluate electronic input submitted by a learner, the reviewer and the learner being users of the electronic learning system, the electronic input comprising an evaluative submission;
divide the initial feedback text data submitted by the reviewer into a plurality of portions;
detect a sentiment associated with each portion of the initial feedback text data submitted by the reviewer;
identify a plurality of sentiment groups in the initial feedback text data submitted by the reviewer, each sentiment group consisting of one or more portions of the initial feedback text data associated with a common sentiment;
select at least one feedback processing module based on the one or more portions of the initial feedback text data corresponding to the sentiment group, the at least one feedback processing module comprising at least one machine-learned model;
for each sentiment group, apply the at least one machine-learned model of the at least one feedback processing module to the corresponding one or more portions of the initial feedback text data submitted by the reviewer to determine at least one suggested revision for the portion of the initial feedback text data submitted by the reviewer; and
generate revised feedback text data indicating each suggested revision for the initial feedback text data submitted by the reviewer.

### Claim 12 (depends on 11)

The system of claim 11, wherein the initial feedback text data is received concurrently with the reviewer submitting a feedback submission to the electronic learning system, and the initial feedback text data includes at least a portion of the feedback submission.

### Claim 13 (depends on 11)

The system of claim 11, wherein the initial feedback text data is received from the computing device associated with the reviewer in response to a prompt displayed at the computing device.

### Claim 14 (depends on 11)

The system of claim 11, wherein at least two sentiment groups in the plurality of sentiment groups include a common portion of the initial feedback text data.

### Claim 15 (depends on 11)

The system of claim 11, wherein at least one sentiment group includes adjacent sentences in the initial feedback text data submitted by the reviewer.

### Claim 16 (depends on 11)

The system of claim 11, wherein the at least one processor being operable to identify the plurality of sentiment groups comprises the at least one processor being operable to identify a plurality of emotionally charged words in the initial feedback text data submitted by the reviewer, and wherein at least one sentiment group includes a portion of the initial feedback text data corresponding to at least one of the emotionally charged words.

### Claim 17 (depends on 11)

The system of claim 11, wherein the at least one processor is operable to:
determine an emotion associated with the corresponding one or more portions of the initial feedback text data submitted by the reviewer; and
determine the at least one suggested revision based on the associated emotion.

### Claim 18 (depends on 11)

The system of claim 11, wherein the at least one processor is operable to:
determine a specificity of the corresponding one or more portions of the initial feedback text data submitted by the reviewer; and
determine the at least one suggested revision based on the specificity.

### Claim 19 (depends on 11)

The system of claim 11, wherein the at least one suggested revision is determined based on a reading complexity level of the learner.

### Claim 20 (depends on 11)

The system of claim 11, wherein the at least one suggested revision is determined based on a psychological profile of the learner.
