# US 2022/0284737 A1 · Method of assessing a pilot emotional state

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284737 A1
- Document ID: US20220284737A1
- Status: Published application
- Kind: A1
- Application number: 17/751,037
- Filed: 2022-05-23
- Published: 2022-09-08
- Earliest claimed: 2018-12-11
- Assignees: GE Aviation Systems LLC
- Applicants: GE Aviation Systems Limited
- Inventors: Stefan Alexander Schwindt; Alexander S. Chen; Peter H. Tu
- Web page: https://patentgrep.com/patent/US20220284737A1

## CPC

- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G08B 21/06: indicating a condition of sleep, e.g. anti-dozing alarms
- A61B 5/1128: {using image analysis (A61B5/1127 takes precedence)}
- A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety}
- A61B 5/18: for vehicle drivers {or machine operators}
- A61B 5/6888: {Cabins}
- A61B 5/746: {Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms}
- B64D 45/0053: {using visual equipment, e.g. cameras}
- G10L 25/63: for estimating an emotional state
- A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens}
- A61B 5/6893: {Cars}
- B64D 45/0059: {by communicating emergency situations to ground control or between crew members}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- B64D: EQUIPMENT FOR FITTING IN OR TO AIRCRAFT; FLIGHT SUITS; PARACHUTES; ARRANGEMENT OR MOUNTING OF POWER PLANTS OR PROPULSION TRANSMISSIONS IN AIRCRAFT
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method 140 of assessing an operator emotional state 131 and sending an alert 144 based on the emotional state 131. The method 140 includes tracking 141 during a time period, using at least one sensor 103, 105, 106, 112, 117, one of an image sensor data, voice data or a biometric parameter of an operator. Determining 142, using a controller 120 that is operatively connected to at least one sensor 103, 105, 106, 112, 117, a probability of a likely emotional state 131 from a list of emotional states 131 of an operator based on one of the image sensor data, voice data or the biometric parameter. Comparing 143, using a processor, the probability of one of the likely emotional states 131 of the operator with a baseline emotional state 131 of the operator. Sending 144, using the controller 120, an alert if most likely emotional state deviates from the baseline emotional state by a predetermined threshold.

## Claims

### Claim at position 1

1-20. (canceled)

### Claim 21

A method of assessing an emotional state of a pilot of an aircraft and sending an alert based on the emotional state, the method comprising:
tracking during a time period, using at least one sensor, one of an image sensor data, voice data or a biometric parameter of the pilot;
determining, using a controller that is operatively connected to at least one sensor, a probability of a likely emotional state from a list of emotional states of the pilot based on one of the image sensor data, voice data or the biometric parameter;
comparing, using a processor, the probability of one of the likely emotional states of the pilot with a baseline emotional state of the pilot; and
sending, using the controller, an alert if the most likely emotional state deviates from the baseline emotional state by a predetermined threshold.

### Claim 22 (depends on 21)

The method of claim 21, wherein the baseline emotional state is generated over time as the pilot flies various flights.

### Claim 23 (depends on 21)

The method of claim 21, further comprising adjusting, using the controller, a workload of the pilot if the most likely emotional state deviates from the baseline emotional state by a predetermined threshold.

### Claim 24 (depends on 21)

The method of claim 21, wherein a sum of each probability for each emotional state equals 100%.

### Claim 25 (depends on 24)

The method of claim 24, further comprising selecting from the probability for each emotional state, using the controller, a highest-probability as the pilot's likely emotional state.

### Claim 26 (depends on 24)

The method of claim 24, wherein the probability for each emotional state ranges from 0-100%.

### Claim 27 (depends on 26)

The method of claim 26, further comprising selecting from the probability for each emotional state, using the controller, the highest-probability as the pilot's likely emotional state.

### Claim 28 (depends on 21)

The method of claim 21, wherein the image sensor data is indicative of one of a pilot's eye movements or body movements.

### Claim 29 (depends on 21)

The method of claim 21, wherein the biometric parameter is based on one of a pilot's breathing, heart rate, or pulse rate.

### Claim 30 (depends on 21)

The method of claim 21, wherein the voice data is indicative of a pilot's voice volume, tone, and spoken content.

### Claim 31 (depends on 21)

The method of claim 21, wherein the step of comparing the probability of one of the likely emotional states of the pilot with a baseline emotional state of the pilot occurs real-time during a flight.

### Claim 32 (depends on 21)

The method of claim 21, wherein the step of comparing the probability of one of the likely emotional states of the pilot with a baseline emotional state of the pilot occurs before a flight has departed.

### Claim 33 (depends on 21)

The method of claim 21, wherein the time period comprises an elapsed time since a pilot has been on-board or an elapsed time of a flight phase.

### Claim 34 (depends on 21)

The method of claim 21, further comprising the step of evaluating at least one of the image sensor data, voice data or a biometric parameter of a pilot to determine a state of intoxication.

### Claim 35 (depends on 21)

The method of claim 21, further comprising the step of evaluating at least one of the image sensor data, voice data or a biometric parameter of a pilot to determine a state of medical emergency.

### Claim 36

A method of assessing an emotional state of a pilot of an aircraft and sending an alert based on the emotional state, the method comprising:
tracking during a time period, using at least one sensor, one of an image sensor data, voice data or a biometric parameter of the pilot;
determining, using a controller that is operatively connected to at least one sensor, a probability of a likely emotional state from a list of emotional states of the pilot based on one of the image sensor data, voice data or the biometric parameter;
comparing, using a processor, the probability of one of the likely emotional states of the pilot with a baseline emotional state of the pilot; and
adjusting, using the controller, a workload of the pilot if the most likely emotional state deviates from the baseline emotional state by a predetermined threshold.

### Claim 37 (depends on 36)

The method of claim 36, wherein the baseline emotional state is generated over time as the pilot flies various flights.

### Claim 38 (depends on 36)

The method of claim 36, wherein the image sensor data is indicative of one of a pilot's eye movements or body movements.

### Claim 39 (depends on 36)

The method of claim 36, wherein the biometric parameter is based on one of a pilot's breathing, heart rate, or pulse rate.

### Claim 40 (depends on 36)

The method of claim 36, wherein the voice data is indicative of a pilot's voice volume, tone, and spoken content.
