# US 2019/0174190 A1 · Apparatus, systems and methods for generating an emotional-based content recommendation list

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0174190 A1
- Document ID: US20190174190A1
- Status: Published application
- Kind: A1
- Application number: 16/127,814
- Filed: 2018-09-11
- Published: 2019-06-06
- Earliest claimed: 2017-12-06
- Assignees: EchoStar Technologies L.L.C.
- Applicants: EchoStar Technologies L.L.C.
- Inventors: Nicholas Newell; Prakash Subramanian
- Web page: https://patentgrep.com/patent/US20190174190A1

## CPC

- H04N 21/4667: {Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections}
- A61B 5/163: {by tracking eye movement, gaze, or pupil change}
- A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety}
- A61B 5/374: Detecting the frequency distribution of signals, e.g. detecting delta, theta, alpha, beta or gamma waves
- A61B 5/378: Visual stimuli
- A61B 5/6803: {Head-worn items, e.g. helmets, masks, headphones or goggles}
- H04N 21/42201: {biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user (input arrangements for interaction with the human body based on nervous system activity detection G06F3/015)}
- H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)}
- H04N 21/4532: {involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)}
- H04N 21/8405: represented by keywords
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

Media content recommendation systems and methods are operable to recommend one or more media content events to a user based on identified changes in the user's emotional state during a life event that is experienced by the user.

## Claims

### Claim 1

A method for recommending media content to a user based on emotions of the user, the method comprising:
receiving Electroencephalography (EEG) information corresponding to a plurality of brain wave lines of a user;
receiving image information captured by an image capture device that is oriented in a direction of a visual field of view of the user while the EEG information is acquired, wherein the EEG information and the image information are concurrently acquired;
identifying an emotional state change of the user based on the received EEG information;
identifying a candidate object in the captured image information, wherein the candidate object corresponds to a physical object seen by the user at a time when the emotional state change of the user was identified; and
identifying one of the plurality of media content events as a recommended media content event based on the candidate object that was identified in the image information.

### Claim 2 (depends on 1)

The method of claim 1, wherein after identifying the at least one candidate object, the method further comprising:
defining an emotional statement based on the identified emotional state change of the user and the identified candidate object, wherein the defined emotional statement is a textual statement that describes the identified candidate object, and wherein the emotional statement comprises a keyword that describes the identified candidate object; and
comparing the keyword of the emotional statement with a plurality of media content event keywords, wherein each of the media content event keyword are associated with one or more of a plurality of media content events,
wherein one of the plurality of media content events is identified as the recommended media content event in response to the media content event keyword matching the keyword of the emotional statement.

### Claim 3 (depends on 1)

The method of claim 1, wherein after identifying the emotional state change of the user based on the received EEG information, the method further comprising:
determining that an emotional state of the user is anticipation when the emotional state change of the user indicates that the user has an emotional feeling of excitement about an event that is going to happen, or is likely to happen, in the near future as a result of experiencing a first type of life event;
determining that the emotional state of the user is surprise when the emotional state change of the user indicates that the user has an emotional feeling of wonder, astonishment, or amazement as a result of experiencing a second type of life event that was unanticipated or that occurred unexpectedly;
determining that the emotional state of the user is joy when the emotional state change of the user indicates that the user has an emotional feeling of well-being, success, good fortune, or by the prospect of possessing what one desires as a result of experiencing a third type of life event;
determining that the emotional state of the user is sadness when the emotional state change of the user indicates that the user has an emotional feeling characterized by feelings of disadvantage, loss, despair, grief, helplessness, disappointment and sorrow as a result of experiencing a fourth type of life event;
determining that the emotional state of the user is disgust when the emotional state change of the user indicates that the user has an emotional feeling of revulsion or profound disapproval aroused by something unpleasant or offensive;
determining that the emotional state of the user is trust when the emotional state change of the user indicates that the user has an emotional feeling that someone is good and honest and will not harm the user, or that someone or something may be relied upon as a result of experiencing a sixth type of life event;
determining that the emotional state of the user is anger when the emotional state change of the user indicates that the user has an emotional feeling characterized by a strong feeling of annoyance, displeasure, or hostility as a result of experiencing a seventh type of life event;
determining that the emotional state of the user is fear when the emotional state change of the user indicates that the user has an emotional feeling of an unpleasant anticipation or awareness of danger, pain, or harm as a result of experiencing an eighth type of life event;
generating a media content event recommendation,
wherein the recommended media content event is indicated to the user as a media content event that the user is likely to enjoy in response to determining that the emotional state of the user is anticipation, joy, surprise or trust,
wherein the recommended media content event is indicated to the user as a media content event that the user is likely to dislike in response to determining that the emotional state of the user is sadness, disgust, anger or fear, and
wherein the recommended media content event is indicated to the user as a media content event that the user is likely to be disinterested in when determining that the emotional state of the user is not one of anticipation, surprise, joy, sadness, disgust, trust, anger, or fear; and
presenting the media content event recommendation to the user.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
generating a user query that asks the user to specify their emotional state during the emotional state change of the user;
comparing the user's specified emotional state to the determined emotional state;
confirming that the determined emotional state is correctly determined when the determined emotional state is the same as the user's specified emotional state; and
changing the determined emotional state to the user's specified emotional state when the determined emotional state is different from the user's specified emotional state.

### Claim 5 (depends on 1)

The method of claim 1, wherein the candidate object identified in the captured image information is a first candidate object, wherein a second candidate object is also identified in the captured image information, and wherein prior to identifying one of the plurality of media content events as the recommended media content event, the method further comprising:
presenting a first object image to the user, wherein the first object image includes the first candidate object;
presenting a second object image to the user, wherein the second object image includes the second candidate object; and
receiving a response from the user indicating that viewing one of the first candidate object or the second candidate object precipitated the emotional state change of the user,
wherein identifying one of the plurality of media content events as the recommended media content event is based on the indicated one of the first candidate object or the second candidate object.

### Claim 6 (depends on 4)

The method of claim 4, further comprising:
generating a user query that asks the user to select one of the first candidate object or the second candidate object as precipitating the change in the user's emotional state during the emotional state change of the user,
wherein the response from the user is a selection of one of the first candidate object or the second candidate object.

### Claim 7 (depends on 1)

The method of claim 1, wherein after the candidate object is identified in the captured image information and before the recommended media content event is identified, the method further comprising:
presenting an image of the candidate object to the user;
generating a user query that asks the user to specify an emotional state that they experienced during the identified emotional state change of the user; and
receiving a user response that specified their emotional state that they experienced during the emotional state change,
wherein one of the plurality of media content events is identified as the recommended media content event based on the user specified emotional state that they experienced during the emotional state change.

### Claim 8 (depends on 1)

The method of claim 1, wherein after the candidate object is identified in the captured image information and before the recommended media content event is identified, the method further comprising:
presenting an image of the candidate object to the user;
generating a user query that asks the user to confirm that the candidate object caused the identified emotional state change of the user; and
receiving a user response that confirms that the candidate object caused the identified emotional state change of the user, wherein the recommended media content event is identified based on the confirmed candidate object.

### Claim 9 (depends on 8)

The method of claim 8, wherein the candidate object identified in the captured image information is a first candidate object, and wherein the received user response does not confirm that the candidate object caused the identified emotional state change of the user, the method further comprising:
identifying a second candidate object in the captured image information when the emotional state change of the user is identified;
generating a user query that asks the user to confirm that the second candidate object caused the identified emotional state change of the user; and
receiving a user response that confirms that the second candidate object caused the identified emotional state change of the user, wherein the recommended media content event is identified based on the confirmed second candidate object.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
receiving eye orientation information acquired by an eye orientation sensor that senses eye orientation of the user while the image information is acquired by the image capture device, wherein the eye orientation information indicates an orientation of at least one eye of the user,
wherein identifying the candidate object in the captured image information is based on the orientation of the at least one eye of the user that indicates which one of a plurality of objects the user was looking at that were within the visual field of view of the user when the image information was acquired.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
receiving audio information acquired by a microphone that includes sounds heard by the user while the image information is acquired by the image capture device; and
identifying a candidate sound in the audio information, wherein the candidate sound corresponds to a sound heard by the user during the emotional state change of the user,
wherein identifying one of the plurality of media content events as a recommended media content event is based on the identified candidate sound.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
generating a user query that asks the user to confirm that the candidate sound precipitated the emotional state change of the user; and
receiving a user response that confirms that the sound caused the identified emotional state change of the user, wherein the recommended media content event is identified based on the confirmed second candidate object.

### Claim 13 (depends on 1)

The method of claim 1, further comprising:
receiving audio information acquired by a microphone that includes sounds heard by the user while the image information is acquired by the image capture device;
identifying a candidate sound in the audio information, wherein the candidate sound corresponds to a sound heard by the user during the emotional state change of the user;
generating a user query that asks the user to select one of the candidate object or the candidate sound as precipitating the change in the user's emotional state during the emotional state change of the user; and
receiving a user response from the user that is a selection of one of the candidate object or the candidate sound,
wherein identifying one of the plurality of media content events as a recommended media content event is based on the identified candidate object when the user selects the candidate object, and
wherein identifying one of the plurality of media content events as a recommended media content event is based on the identified candidate sound when the user selects the candidate sound.

### Claim 14 (depends on 1)

The method of claim 1, wherein after the candidate object is identified in the captured image information and before the recommended media content event is identified, the method further comprising:
presenting an image of the candidate object to the user;
generating a user query that asks the user to specify if an emotional state that they experienced during the identified emotional state change of the user occurred during a real life experience or a content experience, wherein the real live experience occurs during a real life event experienced by the user, and wherein the content experience occurs while the user is viewing a media content event; and
receiving a user response that specifies one of the real life experience and the content experience, wherein the recommended media content event is identified based on the user specified one of the real life experience and the content experience.

### Claim 15 (depends on 1)

The method of claim 1, wherein the EEG information includes first time information that defines time of acquisition of the EEG information, and wherein the image information includes second time information that defines time of acquisition of the image information by the image capture device, the method further comprising:
determining a first time from the first time information, wherein the first time corresponds to an onset of the emotional state change of the user; and
determining a second time from the second time information, wherein the second time is the same as the first time,
wherein the candidate object is identified from the image information that was acquired at the second time.

### Claim 16 (depends on 15)

The method of claim 15, wherein determining the second time from the second time information further comprises:
adding a perception time duration to the first time from the first time information to determine the second time,
wherein the second time precedes the first time that corresponds to the onset of the emotional state change of the user by the perception time duration.

### Claim 17 (depends on 1)

The method of claim 1, further comprising:
determining whether the user was experiencing a real life experience or a content experience during the emotional state change of the user, wherein the real live experience occurs during a real life event experienced by the user, and wherein the content experience occurs while the user is viewing a media content event; and
determining, in response to determining that the user was experiencing the content experience, an identifier of the media content event that the user was viewing,
wherein the recommended media content event is identified based on the determined identifier of the media content event that the user was viewing during the emotional state change of the user.

### Claim 18

A device, comprising:
a transceiver or connector that communicatively couples an emotional state monitor (ESM) to the device,
wherein the ESM communicates Electroencephalograph (EEG) information corresponding to a plurality of brain wave lines of a user being monitored by the ESM, and
wherein the ESM communicates image information captured by an image capture device of the ESM that is oriented in a direction of a visual field of view of the user while the EEG information is acquired, and
wherein the EEG information and the image information are concurrently acquired;
a memory that stores the EEG information and the image information; and
a processor system, wherein the processor system is configured to:
identify an emotional state change of the user based on the received EEG information;
identify a candidate object in the captured image information, wherein the candidate object corresponds to a physical object seen by the user at a time when the emotional state change of the user was identified; and
identify one of the plurality of media content events as a recommended media content event based on the candidate object that was identified in the image information.

### Claim 19 (depends on 18)

The device of claim 18, wherein the processor system is further configured to:
define an emotional statement based on the identified emotional state change of the user and the identified candidate object, wherein the defined emotional statement is a textual statement that describes the identified candidate object, and wherein the emotional statement comprises a keyword that describes the identified candidate object;
retrieve a plurality of media content event keywords that are associated with one or more of a plurality of media content events; and
compare the keyword of the emotional statement with the plurality of media content event keywords, wherein each of the media content event keyword,
wherein one of the plurality of media content events is identified as the recommended media content event in response to the media content event keyword matching the keyword of the emotional statement.

### Claim 20 (depends on 18)

The device of claim 18, wherein the processor system is further configured to:
receive eye orientation information acquired by an eye orientation sensor of the ESM, wherein the eye orientation sensor senses eye orientation of the user while the image information is acquired by the image capture device, wherein the eye orientation information indicates an orientation of at least one eye of the user,
wherein the candidate object in the captured image information is identified based on the orientation of the at least one eye of the user that indicates which one of a plurality of objects the user was looking at that were within the visual field of view of the user when the image information was acquired.
