Pilot cognitive inference system and method

A method for determining pilot cognitive states, comprising:

• receiving, from a plurality of sensors, physiological data from a pilot;
• receiving, from an aircraft state data generator, aircraft state data;
• determining a cognitive state of the pilot based on the physiological data;
• determining available attentional resources of the pilot based on the cognitive state;
• determining attention allocation of the pilot based on the available attentional resources and gaze patterns derived from the physiological data, wherein determining the attention allocation comprises projecting a three-dimensional probabilistic view cone based on the gaze patterns onto a two-dimensional surface to calculate a probability of gaze intersection with each of a plurality of cockpit objects;
• determining situational awareness of the pilot based on the attention allocation and the aircraft state data, wherein determining the situational awareness comprises tracking, using a Probabilistic Graphical Model (PGM) algorithm, a time elapsed since each of a plurality of flight instruments was last observed by the pilot and changes in readings of each of the plurality of flight instruments since the last observation;
• and
• generating a visualization of the available attentional resources of the pilot, the attention allocation of the pilot, the situational awareness of the pilot, and aircraft state data.

A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:

• determine a cognitive state of a pilot based on physiological data using a Gaussian Mixture Model (GMM) that combines outputs from a plurality of Kalman filters, wherein each Kalman filter of the plurality of Kalman filters represents a different hypothesis about the cognitive state of the pilot;
• determine available attentional resources of the pilot based on the cognitive state;
• determine attention allocation of the pilot based on the available attentional resources and gaze patterns derived from the physiological data;
• determine situational awareness of the pilot based on the attention allocation and aircraft state data;
• and
• generate a visualization of the available attentional resources of the pilot, the attention allocation of the pilot, the situational awareness of the pilot, and the aircraft state data.