# US 11,112,613 B2 · Integrated augmented reality head-mounted display for pupil steering > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,112,613 B2 - Document ID: US11112613B2 - Status: Granted - Kind: B2 - Application number: 16/222,789 - Filed: 2018-12-17 - Published: 2019-06-20 - Granted: 2021-09-07 - Earliest claimed: 2017-12-18 - Assignees: Facebook Technologies, LLC - Applicants: Facebook Technologies, LLC - Inventors: Douglas Robert Lanman - Web page: https://patentgrep.com/patent/US11112613B2 ## CPC - G02B 27/0179: {Display position adjusting means not related to the information to be displayed} - G02B 26/101: {with both horizontal and vertical deflecting means, e.g. raster or XY scanners (colour television using laser beams scanning a display screen H04N9/3129)} - G02B 26/105: {with one or more pivoting mirrors or galvano-mirrors (G02B26/101 takes precedence)} - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/0172: {characterised by optical features} - G06F 1/163: {Wearable computers, e.g. on a belt} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye} - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A head-mounted display device for providing augmented reality contents to a wearer includes an eye tracker, a light projector, a beam steerer and a combiner. The eye tracker is configured to determine a position of a pupil of an eye of the wearer. The light projector is configured to project light for rendering images. The beam steerer is configured to change a direction of the light from the light projector based on the position of the pupil. The combiner is configured to combine the light from the light projector and light from an outside of the head-mounted display device for providing an overlap of the rendered image and a real image that corresponds to the light from the outside of the head-mounted display device. ## Claims ### Claim 1 A head-mounted display device for providing augmented reality contents to a wearer, the device comprising: an eye tracker configured to determine a position of a pupil of an eye of the wearer; a light projector configured to project light for rendering images based at least on the augmented reality contents; a beam steerer configured to change a direction of the light from the light projector based on the position of the pupil; and a combiner configured to combine the light from the light projector and light from an outside of the head-mounted display device for providing an overlap of the rendered image and a real image that corresponds to the light from the outside of the head-mounted display device, wherein: the beam steerer is mechanically coupled with the combiner to rotate or translate the combiner so that the light from the light projector impinges on different locations on the combiner, thereby changing the direction of the light from the light projector; and the combiner is configured to reflect the light from the light projector and transmit the light from the outside of the head-mounted display device. ### Claim 2 (depends on 1) The device of claim 1, wherein: the light projector is configured to project the light over an area that is within a 6 mm×6 mm square area on a plane of the pupil of the eye of the wearer. ### Claim 3 (depends on 1) The device of claim 1, wherein: the light projector is configured to project the light over an area with a characteristic dimension of at least 3 mm. ### Claim 4 (depends on 1) The device of claim 1, wherein: the combiner does not include a grating coupled with a waveguide. ### Claim 5 (depends on 1) The device of claim 1, wherein: the light projector includes one or more of: a spatial light modulator or a scanning mirror projector. ### Claim 6 (depends on 1) The device of claim 1, wherein: the light projector is coupled with one or more adjustable focus lenses. ### Claim 7 (depends on 1) The device of claim 1, wherein: the light projector is coupled with an aperture and one or more lenses. ### Claim 8 (depends on 1) The device of claim 1, wherein: the beam steerer includes one or more actuators. ### Claim 9 (depends on 1) The device of claim 1, wherein: the combiner includes one or more of: a Fresnel combiner, a pancake combiner, an ellipsoidal mirror, one or more tunable waveguides, or a holographic combiner. ### Claim 10 (depends on 1) The device of claim 1, wherein: the eye tracker includes one or more of: a pupil image tracker, a retinal reflex tracker, a depth sensor, or a glint tracker. ### Claim 11 (depends on 1) The device of claim 1, wherein: the combiner includes one or more photopolymer films or one or more analog holographic films with a plurality of portions configured to redirect light in distinct directions. ### Claim 12 (depends on 1) The device of claim 1, wherein: the combiner includes an optical waveguide coupled with a plurality of individually adjustable tunable optical elements, a respective individually-adjustable tunable optical element of the plurality of individually adjustable tunable optical elements is configured to have a first optical thickness under a first operation condition and a second optical thickness that is distinct from the first optical thickness under a second operation condition that is distinct from the first operation condition. ### Claim 13 (depends on 1) The device of claim 1, wherein: the eye tracker is integrated with the light projector; the light projector is further configured to project an eye-tracking light; and the combiner is further configured to redirect the eye-tracking light projected by the light projected toward the pupil of the eye of the wearer and redirect the eye-tracking light reflected off a surface of the eye of the wearer toward the eye tracker. ### Claim 14 A method providing augmented reality contents to a wearer using a head-mounted display device that includes an eye tracker, a light projector, a beam steerer, and a combiner, the method comprising: determining, with the eye tracker, a position of a pupil of an eye of the wearer; projecting, with the light projector, light for rendering images based at least on the augmented reality contents; changing, with the beam steerer, a direction of the light from the light projector based on the position of the pupil; and combining, with the combiner, the light from the light projector and light from an outside of the head-mounted display device for providing an overlap of the rendered image and a real image that corresponds to the light from the outside of the head-mounted display device, wherein: the beam steerer is mechanically coupled with the combiner to rotate or translate the combiner so that the light from the light projector impinges on different locations on the combiner, thereby changing the direction of the light from the light projector; and the combiner is configured to reflect the light from the light projector and transmit the light from the outside of the head-mounted display device. ### Claim 15 (depends on 14) The method of claim 14, including: projecting, with the light projector, the light over an area that is within a 6 mm×6 mm square area on a plane of the pupil of the eye of the wearer. ### Claim 16 (depends on 14) The method of claim 14, including: projecting, with the light projector, the light over an area with a characteristic dimension of at least 3 mm. ### Claim 17 (depends on 14) The method of claim 14, wherein: the combiner does not include a grating coupled with a waveguide. ### Claim 18 (depends on 14) The method of claim 14, wherein: the light projector includes one or more adjustable focus lenses. ### Claim 19 (depends on 14) The method of claim 14, wherein: the light projector includes an aperture and one or more lenses.