Method and device for eye tracking with content-embedded glints

A device comprising:

• one or more processors;
• a non-transitory memory;
• an image sensor;
• one or more AR/VR displays;
• and
• one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
• synthesize an AR/VR content stream including a plurality of content frames by embedding a plurality of glints provided for eye tracking into the AR/VR content stream;
• display, via the one or more AR/VR displays, the AR/VR content stream to a user of the HMD;
• obtain, via the image sensor, light intensity data corresponding to an eye of the user of the HMD having projected thereon the plurality of glints;
• and
• determine an orientation of the eye of the user of the HMD based on the light intensity data,
• wherein embedding the plurality of glints provided for eye tracking into the AR/VR content stream includes interleaving an eye tracking frame with the plurality of glints embedded thereon among the plurality of content frames according to a ratio of content frames to eye tracking frames.

A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device with one or more AR/VR displays and an image sensor, cause the device to:

• synthesize an AR/VR content stream including a plurality of content frames by embedding a plurality of glints provided for eye tracking into the AR/VR content stream;
• display, via the one or more AR/VR displays, the AR/VR content stream to a user of the HMD;
• obtain, via the image sensor, light intensity data corresponding to an eye of the user of the HMD having projected thereon the plurality of glints;
• and
• determine an orientation of the eye of the user of the HMD based on the light intensity data,
• wherein embedding the plurality of glints provided for eye tracking into the AR/VR content stream includes interleaving an eye tracking frame with the plurality of glints embedded thereon among the plurality of content frames according to a ratio of content frames to eye tracking frames.