Heads up display with inertial measurement correction for accurate eye tracking

A heads-up display (HUD) system for a vehicle, the HUD system comprising:

• a HUD configured to project augmented reality (AR) graphics on a surface of the vehicle in an active field of view of the driver;
• a driver monitoring camera (DMC) configured to monitor a driver eye position and gaze direction;
• an inertial measurement unit (IMU) configured to measure motion of the vehicle;
• and
• a control system including a computing device having one or more processors and a non-transitory computer-readable storage medium having a plurality of instructions stored thereon, which, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
• receiving driver eye position and gaze direction data from the DMC;
• receiving vehicle motion data from the IMU;
• generating AR HUD graphics based on the driver eye position and gaze direction data and the vehicle motion data;
• and
• displaying, by the HUD, the generated AR HUD graphics on the vehicle surface,
• wherein the control system further processes data from the DMC utilizing Kalman filters to smooth noisy image data and predict an accurate eye gaze position.

A computer-implemented method for controlling a vehicle heads-up display (HUD) system having a HUD configured to project augmented reality (AR) graphics on a surface of the vehicle in an active field of view of the driver, a driver monitoring camera (DMC) configured to monitor a driver eye position and gaze direction, and an inertial measurement unit (IMU) configured to measure motion of the vehicle, the method comprising:

• receiving, at a computing device having one or more processors, driver eye position and gaze direction data from the DMC;
• receiving, at the computing device, vehicle motion data from the IMU;
• generating, by the computing device, AR HUD graphics based on the driver eye position and gaze direction data and the vehicle motion data;
• and
• displaying, by the HUD, the generated AR HUD graphics on the vehicle surface,
• wherein the HUD system further includes a CAN bus connection between the IMU and the control system, wherein data from the IMU is received at a frequency no slower than 30 Hz.