# US 2021/0241654 A1 · Quantitative pilot evaluation during aircraft operation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241654 A1
- Document ID: US20210241654A1
- Status: Published application
- Kind: A1
- Application number: 16/777,651
- Filed: 2020-01-30
- Published: 2021-08-05
- Earliest claimed: 2020-01-30
- Assignees: The Boeing Company
- Applicants: The Boeing Company
- Inventors: Can Onur; Pieter van Gils; Barend-Jan van Bruchem
- Web page: https://patentgrep.com/patent/US20210241654A1

## CPC

- G09B 19/165: {Control of aircraft}
- G05D 1/0061: {for transition from automatic pilot to manual pilot and vice versa}
- A61B 5/18: for vehicle drivers {or machine operators}
- G09B 9/32: by projected image ({G09B9/307,} G09B9/36 take precedence)
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

A device for pilot evaluation during aircraft operation includes a memory, an interface, and one or more processors. The memory is configured to store at least one computational model of at least one human sensory system. The interface is configured to receive sensor data and aircraft state data of an aircraft. The sensor data includes pilot activity data and motion data. The one or more processors are configured to process the motion data and the pilot activity data based on the at least one computational model to predict a pilot estimated aircraft state. The one or more processors are configured to determine an estimated error based on a comparison of the pilot estimated aircraft state and a detected aircraft state. The aircraft state data indicates the detected aircraft state. The one or more processors are configured to selectively activate a notification based, at least in part, on the estimated error.

## Claims

### Claim 1

A device for pilot evaluation during aircraft operation, the device comprising:
a memory configured to store at least one computational model of at least one human sensory system;
an interface configured to receive sensor data and aircraft state data of an aircraft, wherein the sensor data comprises pilot activity data and motion data; and
one or more processors configured to:
process the motion data and the pilot activity data based on the at least one computational model to predict a pilot estimated aircraft state;
determine an estimated error based on a comparison of the pilot estimated aircraft state and a detected aircraft state, wherein the aircraft state data indicates the detected aircraft state; and
selectively activate a notification based, at least in part, on the estimated error.

### Claim 2 (depends on 1)

The device of claim 1, wherein the pilot activity data indicates pilot monitoring activity, pilot control activity, or both.

### Claim 3 (depends on 1)

The device of claim 1, wherein the at least one computational model comprises a vestibular model, a visual model, a proprioceptive model, or a combination thereof.

### Claim 4 (depends on 1)

The device of claim 1, wherein the motion data comprises angular motion data, linear motion data, or both.

### Claim 5 (depends on 1)

The device of claim 1, wherein the detected aircraft state comprises a detected lateral position, a detected longitudinal position, a detected altitude, a detected attitude, a detected linear motion, a detected angular motion, or a combination thereof, of the aircraft.

### Claim 6 (depends on 1)

The device of claim 1, wherein the aircraft state data indicates that the aircraft has the detected aircraft state at a first time, and wherein the one or more processors are configured to predict the pilot estimated aircraft state based, at least in part, on determining that the pilot activity data indicates that an aircraft state indicator is in a line-of-sight of a pilot at a second time and that the second time is within a threshold duration of the first time.

### Claim 7 (depends on 6)

The device of claim 6, wherein the aircraft state indicator comprises an instrument display, an external environmental feature, or both.

### Claim 8 (depends on 7)

The device of claim 7, wherein the instrument display comprises an altimeter, an attitude indicator, a heading indicator, an airspeed indicator, a turn coordinator, a vertical speed indicator, a latitude indicator, a longitude indicator, or a combination thereof.

### Claim 9 (depends on 7)

The device of claim 7, wherein the external environmental feature comprises a landmark, one or more stars, a moon, a horizon, or a combination thereof.

### Claim 10 (depends on 1)

The device of claim 1, wherein the memory is further configured to store an aircraft dynamics model, and wherein the one or more processors are configured to process one or more of the aircraft state data, the motion data, or the pilot activity data based on the aircraft dynamics model to predict the pilot estimated aircraft state.

### Claim 11 (depends on 1)

The device of claim 1, wherein the memory is further configured to store a pilot workload model, and wherein the one or more processors are further configured to process one or more of the aircraft state data, the motion data, or the pilot activity data based on the pilot workload model to determine whether to activate the notification.

### Claim 12 (depends on 1)

The device of claim 1, wherein activating the notification activates autopilot control of the aircraft, sends an alert to an off-board system, provides an alert to a display, outputs an alert via a speaker, or a combination thereof.

### Claim 13

A method of pilot evaluation during aircraft operation, the method comprising:
receiving, at a device, sensor data and aircraft state data of an aircraft, wherein the sensor data comprises pilot activity data and motion data;
processing, at the device, the motion data and the pilot activity data based on at least one computational model of at least one human sensory system to predict a pilot estimated aircraft state;
determining, at the device, an estimated error based on a comparison of the pilot estimated aircraft state and a detected aircraft state, wherein the aircraft state data indicates the detected aircraft state; and
selectively activating a notification based, at least in part, on the estimated error.

### Claim 14 (depends on 13)

The method of claim 13, wherein the pilot activity data comprises first activity sensor data received from a non-contact sensor, second activity sensor data received from a contact sensor, or both.

### Claim 15 (depends on 13)

The method of claim 13, wherein the aircraft state data indicates that the aircraft has the detected aircraft state at a first time, and wherein the pilot estimated aircraft state is predicted based, at least in part, on determining that the pilot activity data indicates that an aircraft state indicator is in a line-of-sight of a pilot at a second time and that the second time is within a threshold duration of the first time.

### Claim 16 (depends on 13)

The method of claim 13, further comprising processing one or more of the aircraft state data, the motion data, or the pilot activity data based on a pilot workload model to determine whether to activate the notification.

### Claim 17

A computer-readable storage device storing instructions that, when executed by one or more processors, causes the one or more processors to initiate, perform, or control operations to:
receive sensor data and aircraft state data of an aircraft, wherein the sensor data comprises pilot activity data and motion data;
process the motion data and the pilot activity data based on at least one computational model of at least one human sensory system to predict a pilot estimated aircraft state;
determine an estimated error based on a comparison of the pilot estimated aircraft state and a detected aircraft state, wherein the aircraft state data indicates the detected aircraft state; and
selectively activate a notification based, at least in part, on the estimated error.

### Claim 18 (depends on 17)

The computer-readable storage device of claim 17, wherein the detected aircraft state comprises a detected lateral position, a detected longitudinal position, a detected altitude, a detected attitude, a detected linear motion, a detected angular motion, or a combination thereof, of the aircraft.

### Claim 19 (depends on 18)

The computer-readable storage device of claim 18, wherein the pilot estimated aircraft state comprises an estimated lateral position, an estimated longitudinal position, an estimated altitude, an estimated attitude, an estimated linear motion, an estimated angular motion, or a combination thereof, of the aircraft.

### Claim 20 (depends on 19)

The computer-readable storage device of claim 19, wherein the estimated error is based on one or more of a difference between the detected lateral position and the estimated lateral position, the detected longitudinal position and the estimated longitudinal position, the detected altitude and the estimated altitude, the detected attitude and the estimated attitude, the detected linear motion and the estimated linear motion, or the detected angular motion and the estimated angular motion.
