Minimal latency apparatus and method for supporting foveated rendering

A method of operating a VR/AR wearable assembly including a wearable frame adapted to be worn over a patient's eyes, a pair of PSOG assembles mounted to the wearable frame with each PSOG assembly including a micro-OLED display, an eye tracking assembly including a plurality of light emitters configured to illuminate an area of a corresponding eye and a plurality of photosensors configured to receive reflected light from different portions of the illuminated corresponding eye, and a substantially disk-shaped pancake optics positioned between the eye tracking assembly and the micro-OLED display, and a processor coupled to each PSOG assembly, the method including the processor performing an algorithm to render computer-generated images on the display device including the steps of:

• determining eye movement tracking based on processing signals from the plurality of photosensors positioned to gather eye movement information;
• detecting a movement of the eye position of the corresponding eye of the patient via a corresponding eye tracking assembly and determining whether the detected eye movement is the beginning of a saccade;
• upon determining the eye movement is the beginning of a saccade, determining an initial saccade gaze position location of the corresponding eye associated with the saccade;
• determining a peak velocity of the saccade;
• determining an amplitude of the saccade based on the determined peak velocity;
• determining a final saccade end gaze position based on the determined peak velocity and the determined amplitude of the saccade;
• and
• rendering a foveated image on a corresponding micro-OLED display at an image location corresponding to the determined final saccade end gaze position.

A non-transitory computer-readable storage media having computer-executable instructions embodied thereon for operating a VR/AR wearable assembly including a wearable frame adapted to be worn over a patient's eyes, a pair of PSOG assemblies mounted to the wearable frame with each PSOG assembly including a micro-OLED display, an eye tracking assembly including a plurality of light emitters configured to illuminate an area of a corresponding eye and a plurality of photosensors configured to receive reflected light from different portions of the illuminated corresponding eye, and a substantially disk-shaped pancake optics positioned between the eye tracking assembly and the micro-OLED display, and a processor coupled to each PSOG assembly, when executed by the processor the computer-executable instructions cause the processor to perform an algorithm to render computer-generated images on the display device including the steps of:

• determining eye movement tracking based on processing signals from the plurality of photosensors positioned to gather eye movement information;
• detecting a movement of the eye position of the corresponding eye of the patient via a corresponding eye tracking assembly and determining whether the detected eye movement is the beginning of a saccade;
• upon determining the eye movement is the beginning of a saccade, determining an initial saccade gaze position location of the corresponding eye associated with the saccade;
• determining a peak velocity of the saccade;
• determining an amplitude of the saccade based on the determined peak velocity;
• determining a final saccade end gaze position based on the determined peak velocity and the determined amplitude of the saccade;
• and
• rendering a foveated image on a corresponding micro-OLED display at an image location corresponding to the determined final saccade end gaze position.