Multisensory gestural-audio interface to promote situational awareness for improved autonomous vehicle control

A non-transitory computer-readable medium having program code recorded thereon for a multisensory gestural-audio interface system, the program code being executed by a processor and comprising:

• program code to provide a haptic intersection representation of a driving environment to a driver of an ego vehicle through a haptics device by pulsing N-times at a center of a driver's palm to indicate N-roads, drawing a line from the center of the palm toward a direction of a clock face position of a road, pulsing at an end of the line and repeating the haptic intersection representation until the driver responds;
• program code to scan, through a multisensory gestural-audio interface device, a vehicle cabin to detect a gesture of the driver in response to the haptic intersection representation of the driving environment;
• program code to provide, through a surrounding ring of speakers of the multisensory gestural-audio interface device, a non-visual, spatial audio description of a selected vehicle control object according to the detected gesture of the driver;
• and
• program code to perform a vehicle control action based on the selected vehicle control object based on a confirmation from the driver to perform the vehicle control action.

A system for a multisensory gestural-audio interface system, the system comprising:

• a non-visual environment overview module to provide a haptic intersection representation of a driving environment to a driver of an ego vehicle through a haptics device by pulsing N-times at a center of a driver's palm to indicate N-roads, drawing a line from the center of the palm toward a direction of a clock face position of a road, pulsing at an end of the line and repeating the haptic intersection representation until the driver responds;
• a gesture tracking module to scan a vehicle cabin to detect a gesture of the driver in response to the haptic intersection representation of the driving environment;
• a vehicle control selection module to provide, through a surrounding ring of speakers of the multisensory gestural-audio interface device, a non-visual, spatial audio description of a selected vehicle control object according to the detected gesture of the driver;
• and
• a vehicle control action module to perform a vehicle control action based on the selected vehicle control object based on a confirmation from the driver to perform the vehicle control action.