# US 11,122,256 B1 · Mixed reality system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,122,256 B1 - Document ID: US11122256B1 - Status: Granted - Kind: B1 - Application number: 16/056,198 - Filed: 2018-08-06 - Granted: 2021-09-14 - Earliest claimed: 2017-08-07 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Richard J. Topliss; Thomas M. Gregory; Paul J. Gelsinger-Austin; Richard H. Tsai - Web page: https://patentgrep.com/patent/US11122256B1 ## CPC - H04N 13/344: with head-mounted left-right displays - G02B 26/101: {with both horizontal and vertical deflecting means, e.g. raster or XY scanners (colour television using laser beams scanning a display screen H04N9/3129)} - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/0172: {characterised by optical features} - H04N 9/3129: {scanning a light beam on the display screen (scanning a light beam on a screen in displays other than projection devices G09G3/02; scanning systems in general G02B26/10; projectors using laser light sources in general H04N9/3161)} - H04N 9/3161: {using laser light sources (using laser beams scanning the display screen H04N9/3129)} - H04N 13/383: for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes - H04N 13/398: Synchronisation thereof; Control thereof - G02B 2027/0112: {comprising device for genereting colour display} - G02B 2027/0134: {of stereoscopic type} - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/0174: {holographic} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS ## Abstract A mixed reality direct retinal projector system that may include a headset that uses a reflective holographic combiner to direct light from a light engine into an eye box corresponding to a user's eye. The light engine may include light sources coupled to projectors that independently project light to the holographic combiner from different projection points. The light sources may be in a unit separate from the headset that may be carried on a user's hip, or otherwise carried or worn separately from the headset. Each projector may include a collimating and focusing element, an active focusing element, and a two-axis scanning mirror to project light from a respective light source to the holographic combiner. The holographic combiner may be recorded with a series of point to point holograms; each projector interacts with multiple holograms to project light onto multiple locations in the eye box. ## Claims ### Claim 1 A system, comprising: a controller; a reflective holographic combiner comprising a plurality of point-to-point holograms; a plurality of light sources that emit light beams under control of the controller; and a plurality of projectors, wherein each one of the plurality of projectors includes a respective scanning mirror corresponding to a different one of the plurality of light sources, and wherein the respective scanning mirror of the each one projector scans a light beam emitted by the corresponding different light source to the holographic combiner; wherein the plurality of point-to-point holograms redirect light beams received from the plurality of projectors to a plurality of eye box points, wherein each projector scans the light beam to two or more of the plurality of holograms, wherein the two or more holograms redirect the light beam to illuminate two or more respective eye box points, wherein neighboring eye box points are illuminated by different ones of the projectors. ### Claim 2 (depends on 1) The system as recited in claim 1, wherein the light sources are RGB lasers. ### Claim 3 (depends on 1) The system as recited in claim 1, wherein the scanning mirror of each projector includes a two-axis scanning mirror that scans the light beam emitted by the corresponding different light source to the holographic combiner. ### Claim 4 (depends on 3) The system as recited in claim 3, wherein the scanning mirror is a 2D scanning microelectromechanical systems (MEMS) mirror. ### Claim 5 (depends on 3) The system as recited in claim 3, wherein the scanning mirror performs a non-linear scan so that resolution is higher near a center of an image formed by the scan than in peripheral regions of the image. ### Claim 6 (depends on 3) The system as recited in claim 3, wherein each projector further includes collimating and focusing optics that affect the light beam emitted by the respective light source so that the light beam is substantially collimated when reflected to the eye box points by the holographic combiner. ### Claim 7 (depends on 3) The system as recited in claim 3, wherein each projector further includes an active focusing element that changes focus of the light beam as the light beam is scanned across a horizontal axis of the holographic combiner by the scanning mirror. ### Claim 8 (depends on 1) The system as recited in claim 1, wherein the controller is configured to selectively activate particular ones of the plurality of light sources and respective ones of the plurality of projectors to scan light beams to the holographic combiner from different projection points corresponding to the projectors. ### Claim 9 (depends on 1) The system as recited in claim 1, wherein the system further includes a gaze tracking component that tracks position of a subject's eye, wherein the controller selectively activates and modulates particular ones of the plurality of light sources to selectively illuminate particular ones of the plurality of eye box points based on current position of the subject's eye as determined by the gaze tracking component. ### Claim 10 (depends on 1) The system as recited in claim 1, further comprising: a headset configured to be worn on the head of a user, wherein the headset includes the reflective holographic combiner and the plurality of projectors; and a control box separate from the headset, wherein the control box includes the plurality of light sources; wherein the plurality of light sources are coupled to the plurality of projectors by fiber optic cables. ### Claim 11 (depends on 10) The system as recited in claim 10, wherein the control box further includes the controller and a power source. ### Claim 12 (depends on 1) The system as recited in claim 1, further comprising a headset configured to be worn on the head of a user, wherein the headset includes the reflective holographic combiner, the plurality of projectors, and the plurality of light sources. ### Claim 13 (depends on 1) The system as recited in claim 1, wherein the system comprises a plurality of light sources, a plurality of projectors, and a holographic combiner for each of a subject's eyes, wherein the light sources, projectors, and holographic combiner for a given eye project light to an eyebox corresponding to that eye. ### Claim 14 (depends on 1) The system as recited in claim 1, wherein the holographic combiner is one or more layers of holographic film on either side of or embedded in a lens positioned in front of a subject's eye. ### Claim 15 A method, comprising: emitting, by a plurality of light sources, light beams to a plurality of projectors under control of a controller, each light source corresponding to a different one of the plurality of projectors; scanning, by the plurality of projectors, the light beams to a holographic combiner, wherein each one of the plurality of projectors includes a respective scanning mirror which scans the light beam emitted by the corresponding different one of the plurality of light sources; and redirecting, by a plurality of point-to-point holograms of the holographic combiner, the light beams to respective ones of a plurality of eye box points; wherein each projector scans a light beam to two or more of the plurality of holograms, wherein the two or more holograms redirect the light beam to illuminate two or more respective eye box points, and wherein neighboring eye box points are illuminated by different ones of the projectors. ### Claim 16 (depends on 15) The method as recited in claim 15, wherein the light sources are RGB lasers. ### Claim 17 (depends on 15) The method as recited in claim 15, wherein each projector includes: collimating and focusing optics that affect the light beam emitted by the respective light source so that the light beam is substantially collimated when reflected to the eye box points by the holographic combiner; a two-axis scanning mirror that scans the light beam emitted by the respective light source to the holographic combiner; and an active focusing element that changes focus of the light beam as the light beam is scanned across a horizontal axis of the holographic combiner by the scanning mirror. ### Claim 18 (depends on 15) The method as recited in claim 15, wherein scanning comprises performing a non-linear scan so that resolution is higher near a center of an image formed by the scan than in peripheral regions of the image. ### Claim 19 (depends on 15) The method as recited in claim 15, further comprising selectively activating particular ones of the plurality of light sources and respective ones of the plurality of projectors to scan light beams to the holographic combiner from different projection points corresponding to the projectors. ### Claim 20 (depends on 15) The method as recited in claim 15, further comprising selectively activating and modulating particular ones of the plurality of light sources to selectively illuminate particular ones of the plurality of eye box points based on current position of the subject's eye. ### Claim 21 (depends on 15) The method as recited in claim 15, wherein the reflective holographic combiner and the plurality of projectors are components of a headset configured to be worn on the head of a user, wherein the plurality of light sources are contained in a control box separate from the headset, and wherein the plurality of light sources are coupled to the plurality of projectors by fiber optic cables. ### Claim 22 (depends on 15) The method as recited in claim 15, wherein the reflective holographic combiner, the plurality of projectors, and the plurality of light sources are components of a headset configured to be worn on the head of a user.