Back to results
US 12,612,182 B2Granted
Tuned flight language model for cognitive workload
AssigneeRockwell Collins, Inc.
InventorJason A Myren
Life of this patent
Estimating the term…
Abstract
A pilot notification system includes multiple sensors, multiple modules, and a large language model (LLM). The modules processes inputs to determine the cognitive state of the pilot, the current pilot workload, and the current flight conditions of the aircraft. These modules send this information to other modules which process this information and send the information to the LLM. The structure of the LLM is modified to reflect the pilot's current cognitive state, the pilot's current workload, and the current flight conditions. The LLM generates a pilot alert message to alert the pilot of import information while considering the current circumstances.
Claims20
A pilot notification system comprising:
- a plurality of pilot cognitive state sensors;
- a plurality of pilot interface sensors;
- a plurality of flight conditions sensors;
- a pilot cognitive state module configured to receive pilot cognitive state sensor input from the plurality of pilot cognitive state sensors,
- wherein the pilot cognitive state module processes the pilot cognitive state sensor input to determine a cognitive state of the pilot, generates a pilot cognitive state report that reflects the cognitive state of the pilot, and transmits the pilot cognitive state report to a large language model (LLM) Tuning Controller;
- a pilot workload module configured to receive pilot interface sensor input from the plurality of pilot interface sensors, wherein the plurality of pilot interface sensors are configured to be physically touched by the pilot and audibly interfaced with by the pilot and wherein the plurality of pilot interface sensors are further configured to send the pilot interface sensor input to the pilot workload module,
- wherein the pilot workload module processes the pilot interface sensor input to determine pilot workload, generates a pilot workload report that reflects pilot workload, and transmits the pilot workload report to the LLM Tuning Controller;
- a flight conditions module configured to receive flight conditions sensor input from the plurality of fight conditions sensors,
- wherein the flight conditions module processes the flight conditions sensor input to determine flight conditions of the aircraft, generates a flight conditions report, and transmit the flight conditions report to the LLM Tuning Controller;
- wherein the LLM Tuning Controller includes a LLM Tuner Module and a Tuning Controller Alert Module;
- wherein the LLM Tuning Controller is configured to receive the pilot cognitive state report, the pilot workload report, and the flight conditions report and to cause the LLM Tuner Module to:
- generate a set of tuning modifications based on the pilot cognitive state report, the pilot workload report, and the flight conditions report, wherein the set of tuning modifications cause the LLM Tuner Module to modify the current structure of a LLM to reflect the pilot cognitive state report, the pilot workload report, and the flight conditions report;
- wherein the LLM Tuning Controller is further configured to cause the Tuning Controller Alert Module to:
- generate a tuning controller alert based on at least one of the pilot cognitive state report, the pilot workload report, the flight conditions report;
- transmit the tuning controller alert to the LLM;
- wherein the LLM is configured to receive the tuning controller alert, generates a pilot alert message based on the tuning controller alert, and transmits the pilot alert message to a pilot communications module to present the pilot alert message to a pilot,
- wherein the LLM generates the pilot alert message using the tuning controller alert and an artificial intelligence algorithm that resides in the LLM;
- and
- wherein the artificial intelligence algorithm has been adjusted using the set of tuning modifications.
16 more elements
A method of operating a pilot notification system comprising:
- receiving a pilot observation sensor input to a pilot observation sensor configured to collect data about a pilot from observing the pilot;
- and
- wherein the pilot observation sensor input is any input to the pilot observation sensor;
- transmitting the pilot observation sensor input to a pilot cognitive state module, wherein the pilot cognitive state module is a computer program configured to analyze the pilot observation sensor input to determine a cognitive state of the pilot;
- and
- wherein the pilot's cognitive state includes a mental state of the pilot;
- analyzing the pilot observation sensor input using the pilot cognitive state module to generate a pilot cognitive state report that summarizes the cognitive state of the pilot;
- and
- wherein the pilot cognitive state report includes a first signal to notify the pilot of the pilot cognitive state report;
- transmitting the pilot cognitive state report from the pilot cognitive state module to a LLM Tuning Controller;
- receiving a pilot interface sensor input from a plurality of pilot interface sensors and transmitting the pilot interface sensor input to a pilot workload module configured to determine the pilot workload, wherein the plurality of pilot interface sensors are configured to be physically touched by the pilot and audibly interfaced with by the pilot and wherein the plurality of pilot interface sensors are further configured to send the pilot interface sensor input to the pilot workload module;
- analyzing the pilot observation sensor input using the pilot workload module to generate a pilot workload state report that summarizes the pilot workload;
- and
- transmitting the pilot workload report from the pilot workload module to the LLM Tuning Controller;
- wherein the LLM Tuning Controller also uses the pilot workload report to generate the set of tuning modifications;
- receiving a flight conditions sensor input to a flight conditions sensor configured to collect data about the environment external to the aircraft and data about the flight conditions of an aircraft;
- and
- wherein the flight conditions sensor input is any input to the flight conditions sensor;
- transmitting the flight conditions sensor input to a flight conditions module configured to analyze the flight condition input to determine a set of important flight condition information;
- and
- analyzing the flight conditions sensor input using the flight conditions module to determine the set of important flight condition information and generate a flight conditions report using the set of important flight condition information and
- wherein the flight conditions report includes a second signal to notify the pilot of the flight condition report;
- transmitting the flight conditions report from the flight conditions module to the LLM Tuning Controller;
- receiving the pilot cognitive state report and the flight conditions report in the LLM Tuning Controller;
- generating a set of tuning modifications in the LLM Tuner Module based on the pilot cognitive state report and the flight conditions report,
- wherein the set of tuning modifications cause the LLM Tuner Module to modify the current structure of a LLM to reflect the pilot cognitive state report and the flight conditions report;
- generating a tuning controller alert in the Tuning Alert Module based on at least one of the pilot cognitive state report or the flight conditions report;
- transmitting the tuning controller alert from the Tuning Alert Module to the large language model;
- receiving the tuning controller alert in the LLM;
- generating a message in the LLM using the tuning controller alert and an artificial intelligence algorithm,
- wherein the artificial intelligence algorithm has been adjusted using the set of tuning instructions;
- transmitting the message from the LLM to a pilot communications module;
- and
- displaying the message to a pilot using the pilot communications module.
31 more elements
Dependencies are inferred from the claim text.