Vehicle heads-up display system with augmented reality and vision tracking device

A method of operating a heads-up display system in a vehicle having a screen and a controller with a processor and tangible, non-transitory memory, the method comprising:

• projecting virtual images on the screen, the virtual images being superimposed on a real-world view visible to the operator, via the controller, the virtual images being projected on the screen inside a bounding box;
• obtaining vision data of the operator, via at least one tracking device in communication with controller;
• determining a gaze category of the operator based on the vision data, including a binary determination of whether the operator is gazing at the screen, via the controller;
• identifying a vision classification of the operator based on the vision data, the vision classification including binocular vision and monocular vision, via the controller;
• and
• modifying the virtual images projected on the screen based in part on the gaze category and the vision classification, via the controller, including:
• shifting a position of the bounding box on the screen based on a functioning eye when the operator is identified to have monocular vision;
• and
• increasing a size of the bounding box when the operator is identified to have binocular vision and is not gazing at the screen.

A vehicle comprising:

• a screen in proximity to an operator and sufficiently transparent such that a real-world view is visible to the operator through the screen;
• a heads-up display system having a controller adapted to project virtual images on the screen, the virtual images being superimposed on the real-world view, the controller having a processor and tangible, non-transitory memory on which instructions are recorded;
• at least one tracking device operatively connected to the vehicle and configured to obtain vision data of the operator, the virtual images being projected on the screen inside a bounding box;
• wherein the controller is adapted to receive the vision data and:
• determine a gaze category of the operator based on the vision data, including a binary determination of whether the operator is gazing at the screen;
• identify a vision classification of the operator based on the vision data, the vision classification including binocular vision and monocular vision;
• modify the virtual images projected on the screen based in part on the gaze category and the vision classification;
• and
• wherein modifying the virtual images includes:
• when the operator is identified to have monocular vision, shifting a position of the bounding box on the screen based on a functioning eye;
• and
• when the operator is identified to have binocular vision and is not gazing at the screen, increasing a size of the bounding box.