# US 2024/0203281 A1 · Smart e-learning system using adaptive video lecture delivery based on attentiveness of the viewer

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203281 A1
- Document ID: US20240203281A1
- Status: Published application
- Kind: A1
- Application number: 18/558,114
- Filed: 2022-04-15
- Published: 2024-06-20
- Earliest claimed: 2021-07-18
- Inventors: Payal Doshi
- Web page: https://patentgrep.com/patent/US20240203281A1

## CPC

- G09B 5/06: with both visual and audible presentation of the material to be studied
- G09B 5/065: {Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06V 40/174: {Facial expression recognition}
- G09B 7/00: Electrically-operated teaching apparatus or devices working with questions and answers (mechanically operated G09B3/00; computing arrangements G06F)
- G09B 7/08: characterised by modifying the teaching programme in response to a wrong answer, e.g. repeating the question or supplying further information
- H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement}
- G06N 20/00: Machine learning
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present invention relates to Smart E-Learning system using adaptive video lecture delivery based on attentiveness of the viewer. The attentiveness of the viewer is increased, who is watching a video lecture (204) in an E-learning scenario where viewer's attentiveness score (205) is captured based on facial expressions, cognitive responses etc. The video content (209) are changed dynamically to increase the attentiveness of the viewer in real time without viewer's knowledge. To change content segments, next segment predictor (206) has prior knowledge of the main video with its break points and supporting videos (207). It (206) uses viewer's attentiveness (210) and selects the next segment (209) which is played for the viewer, before proceeding with the main content. Incremental Learning with AI/MML techniques are used for next segment prediction. The feedback of attentiveness score after content segment change is used by the Next Segment Predictor (206) for further learning process.

## Claims

### Claim 1

A Smart E-Learning system using adaptive video lecture delivery based on attentiveness of the viewer, the said system comprising:
a. a webcam or any sensory input (200) for capturing facial expressions or cognitive responses to calculate attentiveness level (210);
b. a client machine (201) as user console comprising:
Video lecture segments (204) played as eLearning contents on display (203); Measure and analyze Attentiveness module (205) wherein the module is a trained model based on A I/ML (Artificial Intelligence/Machine learning) techniques for measuring and analyzing the attentiveness level as continuous feedback (210) of the viewer;
c. a server machine (202) as a backend processing unit comprising:
next segment predictor (206) is an incremental training model based on A I/ML techniques which finds correlation between viewer's attentiveness level and video lecture segment played; using user's previous responses (208), continuous feedback of viewer as its attentiveness level (210), wherein next video content segment (209) is selected by next segment predictor (206) from other related video content segments stored on server machine (202); this selected segment is given as server next (211) response to client machine (201) to be played as next video content.

### Claim 2 (depends on 1)

The system claimed in claim 1 is a process of E-Learning; wherein while the user is watching the online video contents, viewer's attentiveness level is detected based on various parameters like facial expressions, cognitive response mapped based on, but not limited to, eye movement, eye aspect ratio, lip movement, forehead expression etc., these are then compared with a pre-trained model (205) of attentiveness detection based on A I/ML techniques; whereas this step may be carried out either on the client side (201) or on the server-side (202); the final attentiveness level score is then sent to the server (202) for predicting next video segment to be played online; wherein the improvisation in this process is:
video content segments and its sequence are changed and played dynamically in real time on viewer's console based on viewer's attentiveness in order to increase or maintain appropriate attentiveness level of the viewer during E-Learning.

### Claim 3 (depends on 1)

In the system claimed in claim 1, the said server (202) once received the attentiveness score as continuous feedback (210), uses it to find if there should be any change in the content sequence of video lecture; wherein the server (202) already has pre-loaded information (207) about the current topic of the content/lecture as main content video;
the main content video is marked for break points where the new content can be inserted; these break points are marked smartly by the content creator using an authoring tool; wherein the authoring tool has facility to upload the predicted content video with the facility to mark the break point timing in the main content video; each main content video is accompanied by several supporting content segments wherein these segments may consist of, but not limited to, explanatory video with more examples, MCQ (Multiple Choice question) along with their explanatory videos or correct as well as wrong response from the user, solution of certain extra questions, points to remember etc.; each supporting segment is classified into different content type.

### Claim 4 (depends on 1)

In the system claimed in claim 1, the smart decision of what content type should be selected for what kind of attentiveness trend is derived from incremental training model (302) developed with AI/ML like technique as next segment predictor (206), wherein for this the viewer's feedback in terms of attentiveness level (210) for the newly inserted content segment (209) is observed; the change in attentiveness is stored for future reference (208), this data is then fed to the next segment predictor (206) to decide if sending a particular content segment is desirable or not; once this decision is made, new content segments are selected and sent to the client as serve next (211).

### Claim 5 (depends on 3)

The steps claimed in claim 3 can also be run on the client side wherein client is preloaded with all the information about the content segments available for insertion.

### Claim 6 (depends on 1)

In the system claimed in claim 1, the new content segment (209) to be inserted is received on the client side (211); wherein as soon as the next break point is hit the new content segments are played, which happens without any notification to the viewer/user; for the user it seems to be a pre-decided (single video) but in real initial video is continuously being modified adaptively to suit the temperament of the user/viewer.

### Claim 7 (depends on 1)

A process of the system claimed in claim 1 for smart E-Learning using adaptive video lecture delivery based on attentiveness of the viewer, wherein the said process comprising:
While the video segment is playing (101), attentiveness of the viewer is measured using facial expressions, cognitive responses etc. in parallel (102) using the web-camera (100) or any other sensory inputs;
The attentive score is analysed (104);
if the viewer is not attentive or attentiveness is decreasing:
MCQ or some interesting video segment is presented to viewer related to the topic (105); Response for MCQ is recorded (107), if the response is incorrect, an explainer video is played to explain the correct answer (109); else if MCQ is answered correctly; next MCQ/video is played as per availability (108);
if viewer is attentive or attentiveness is increasing:
no changes are done in online content delivery (110);
if viewer is confused or attentiveness remains same:
interesting fact, re-explanation of video with more examples is shown (106); after the said new content segment/MCQ/Video insertion is played (106,108,109,110), the next video segment in the lecture is played by repeating above said process in similar manner (101);
attentiveness trend is captured and sent to the next segment predictor learning model (103) as an ongoing process; this data is stored and used as a reference point to mature the process that will actually be used for predicting the next content segment.
