# US 2014/0154659 A1 · Facial expression recognition in educational learning systems > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0154659 A1 - Document ID: US20140154659A1 - Status: Published application - Kind: A1 - Application number: 14/086,695 - Filed: 2013-11-21 - Published: 2014-06-05 - Earliest claimed: 2012-11-21 - Assignees: Laureate Education, Inc. - Applicants: Laureate Education, Inc. - Inventors: Ken Otwell - Web page: https://patentgrep.com/patent/US20140154659A1 ## CPC - G09B 5/02: with visual presentation of the material to be studied, e.g. using film strip - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) ## Abstract Method, systems, and media for participating in and conducting a learning session of a collaborative network. Various embodiments of methods, systems, and media for participating in a learning session of a collaborative network are presented. User image data is received. The user image data is converted into expressive avatar information comprising an avatar identifier and an avatar emotion identifier. The expressive avatar information is transmitted. Altered instructional content is received, wherein the altered instructional content is an alteration of the instructional content and the alteration is based on a plurality of avatar identifiers and a plurality of avatar emotion identifiers, wherein the plurality of avatar identifiers comprises at least the avatar identifier, and wherein the plurality of avatar emotion identifiers comprises at least the avatar emotion identifier. User image data may be converted into expressive avatar data using facial expression recognition techniques. ## Claims ### Claim 1 A method for participating in a learning session over a collaborative network, the method comprising: receiving user image data, wherein the user image data is associated with a time during a presentation of instructional content; converting the user image data into expressive avatar information, the expressive avatar information comprising an avatar identifier and an avatar emotion identifier; transmitting the expressive avatar information; and receiving altered instructional content, wherein the altered instructional content is an alteration of the instructional content and the alteration is based on a plurality of avatar identifiers and a plurality of avatar emotion identifiers, wherein the plurality of avatar identifiers comprises at least the avatar identifier, and wherein the plurality of avatar emotion identifiers comprises at least the avatar emotion identifier. ### Claim 2 (depends on 1) The method of claim 1, wherein converting the user image data into expressive avatar information further comprises: analyzing the user image data to produce feature data; classifying the feature data to produce an emotion value; assigning the emotion value to the avatar emotion identifier; and associating the avatar emotion identifier with the avatar identifier. ### Claim 3 (depends on 1) The method of claim 1, wherein receiving user image data further comprises: receiving the user image data from an input device, the input device comprising a camera, a video recorder, an integrated camera, a file, a streaming video source, a computer, a portable computer, a mobile device, or a phone. ### Claim 4 (depends on 1) The method of claim 1 further comprising: receiving second expressive avatar information associated with the presentation of instructional content from a plurality of participant workstations, the second expressive avatar information comprising a second plurality of avatar identifiers and a second plurality of avatar emotion identifiers; presenting a student interface comprising an avatar status display; determining a plurality of avatar images based on the second plurality of avatar identifiers and the second plurality of avatar emotion identifiers; and displaying the plurality of avatar images in the avatar status display. ### Claim 5 A method for conducting a learning session over a collaborative network, the method comprising: receiving expressive avatar information associated with a presentation of instructional content from a plurality of participant workstations, the expressive avatar information comprising a plurality of avatar identifiers and a plurality of avatar emotion identifiers; altering the instructional content based on the plurality of avatar identifiers and the plurality of avatar emotion identifiers; and transmitting the altered instructional content to the plurality of participant workstations. ### Claim 6 (depends on 5) The method of claim 5, wherein altering the instructional content further comprises: comparing the plurality of avatar identifiers and the plurality of avatar emotion identifiers to a model; determining whether the expressive avatar information exceeds a threshold of the model; and if the expressive avatar information exceeds the threshold, altering the instructional content by performing an action associated with the threshold. ### Claim 7 (depends on 5) The method of claim 5 further comprising: presenting an instructor interface, the instructor interface comprising a current instructional content display, an avatar status display, and an action interface element associated with an action; displaying the instructional content in the current instructional content display; determining a plurality of avatar images based on the plurality of avatar identifiers and the plurality of avatar emotion identifiers; displaying the plurality of avatar images in the avatar status display; receiving a selection of the action interface element; and wherein altering the instructional content further comprises altering the instructional content by performing the action associated with the action interface element on the instructional content. ### Claim 8 (depends on 7) The method of claim 7 further comprising: displaying a suggestion request interface element on the instructor interface; receiving a selection of the suggestion request interface element; and providing a suggested action to alter the instructional content. ### Claim 9 (depends on 5) The method of claim 5, wherein altering the instructional content further comprises: pausing display of the instructional content; stopping display of the instructional content; replacing the instructional content with a second instructional content; or inserting additional instructional content into the instructional content. ### Claim 10 A system for participating in a learning session over a collaborative network, the system comprising: a processor configured to: receive user image data, wherein the user image data is associated with a time during a presentation of instructional content; convert the user image data into expressive avatar information, the expressive avatar information comprising an avatar identifier and an avatar emotion identifier; transmit the expressive avatar information; receive altered instructional content, wherein the altered instructional content is an alteration of the instructional content and the alteration is based on a plurality of avatar identifiers and a plurality of avatar emotion identifiers, wherein the plurality of avatar identifiers comprises at least the avatar identifier, and wherein the plurality of avatar emotion identifiers comprises at least the avatar emotion identifier. ### Claim 11 (depends on 10) The system of claim 10, wherein the processor is further configured to: analyze the user image data to produce feature data; classify the feature data to produce an emotion value; assign the emotion value to the avatar emotion identifier; and associate the avatar emotion identifier with the avatar identifier. ### Claim 12 (depends on 10) The system of claim 10, wherein the processor is further configured to: receive the user image data from an input device, the input device comprising a camera, a video recorder, an integrated camera, a file, a streaming video source, a computer, a portable computer, a mobile device, or a phone. ### Claim 13 (depends on 10) The system of claim 10, wherein the processor is further configured to: receive second expressive avatar information associated with the presentation of instructional content from a plurality of participant workstations, the second expressive avatar information comprising a second plurality of avatar identifiers and a second plurality of avatar emotion identifiers; present a student interface comprising an avatar status display; determine a plurality of avatar images based on the second plurality of avatar identifiers and the second plurality of avatar emotion identifiers; and display the plurality of avatar images in the avatar status display. ### Claim 14 A system for conducting a learning session over a collaborative network, the system comprising: a processor configured to: receive expressive avatar information associated with a presentation of instructional content from a plurality of participant workstations, the expressive avatar information comprising a plurality of avatar identifiers and a plurality of avatar emotion identifiers; alter the instructional content based on the plurality of avatar identifiers and the plurality of avatar emotion identifiers; and transmit the altered instructional content to the plurality of participant workstations. ### Claim 15 (depends on 14) The system of claim 14, wherein the processor is further configured to: compare the plurality of avatar identifiers and the plurality of avatar emotion identifiers to a model; determine whether the expressive avatar information exceeds a threshold of the model; and alter the instructional content by performing an action associated with the threshold if the expressive avatar information exceeds the threshold. ### Claim 16 (depends on 14) The system of claim 14, wherein the processor is further configured to: present an instructor interface, the instructor interface comprising a current instructional content display, an avatar status display, and an action interface element associated with an action; display the instructional content in the current instructional content display; determine a plurality of avatar images based on the plurality of avatar identifiers and the plurality of avatar emotion identifiers; display the plurality of avatar images in the avatar status display; receive a selection of the action interface element; and wherein altering the instructional content further comprises altering the instructional content by performing the action associated with the action interface element on the instructional content. ### Claim 17 (depends on 16) The system of claim 16, wherein the processor is further configured to: display a suggestion request interface element on the instructor interface; receive a selection of the suggestion request interface element; and provide a suggested action to alter the instructional content. ### Claim 18 (depends on 14) The system of claim 14, wherein the processor is further configured to: pause display of the instructional content; stop display of the instructional content; replace the instructional content with a second instructional content; or insert additional instructional content into the instructional content. ### Claim 19 A computer readable storage medium comprising instructions that if executed enable a computing system to: receive user image data, wherein the user image data is associated with a time during a presentation of instructional content; convert the user image data into expressive avatar information, the expressive avatar information comprising an avatar identifier and an avatar emotion identifier; transmit the expressive avatar information; receive altered instructional content, wherein the altered instructional content is an alteration of the instructional content and the alteration is based on a plurality of avatar identifiers and a plurality of avatar emotion identifiers, wherein the plurality of avatar identifiers comprises at least the avatar identifier, and wherein the plurality of avatar emotion identifiers comprises at least the avatar emotion identifier. ### Claim 20 A computer readable storage medium comprising instructions that if executed enable a computing system to: receive expressive avatar information associated with a presentation of instructional content from a plurality of participant workstations, the expressive avatar information comprising a plurality of avatar identifiers and a plurality of avatar emotion identifiers; alter the instructional content based on the plurality of avatar identifiers and the plurality of avatar emotion identifiers; transmit the altered instructional content to the plurality of participant workstations.