Low-latency fusing of color image data

A system for presenting a mixed reality experience, the system comprising:

• a head mounted display device including a color sequential display for projecting a virtual object using first, second and third color channels;
• a plurality of sensors for sensing positions of the display device, the plurality of sensors comprising an inertial measurement unit for sensing movement of the head mounted display device and at least one image capture device;
• and
• one or more processors for determining a three-dimensional map of the environment in which the system is used based on data from the plurality of sensors, the one or more processors rendering the first, second and third color channels at a time t1, the one or more processors predicting a position of the display device at a time t2, after t1, when the color sequential display is to display the first color channel, the one or more processors predicting a position of the display device at a time t3, after t2, when the color sequential display is to display the second color channel, the one or more processors predicting a position of the display device at a time t4, after t3, when the color sequential display is to display the third color channel, the one or more processors adjusting the image data for the virtual object on the first, second and third color channels, independently of each other, based on the predicted position of the display device at the times t2, t3 and t4.

A method of displaying an image using a display device including a color sequential display, the method comprising:

• (a) determining a view of the display device;
• (b) rendering image data based on the view determined in said step (a);
• (c) predicting an updated view of the display device after said step (a);
• and
• (d) applying one or more transforms to image data for color channels of the color sequential display to adjust the image data for the color channels based on the updated view predicted in said step (c), image data for a first color channel adjusted differently than image data for a second color channel.