# US 2021/0243384 A1 · Imaging system and method for producing images using means for adjusting optical focus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0243384 A1 - Document ID: US20210243384A1 - Status: Published application - Kind: A1 - Application number: 17/239,995 - Filed: 2021-04-26 - Published: 2021-08-05 - Earliest claimed: 2019-07-08 - Assignees: Varjo Technologies Oy - Applicants: Varjo Technologies Oy - Inventors: Mikko Ollila - Web page: https://patentgrep.com/patent/US20210243384A1 ## CPC - H04N 5/2621: {Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability} - G02B 7/09: adapted for automatic focusing or varying magnification - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/017: {Head mounted} - G03B 13/36: Autofocus systems - G03B 30/00: Camera modules comprising integrated lens units and imaging units, specially adapted for being embedded in other devices, e.g. mobile phones or vehicles - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06V 20/20: in augmented reality scenes - G06V 40/18: Eye characteristics, e.g. of the iris - H04N 23/55: Optical parts specially adapted for electronic image sensors; Mounting thereof - H04N 23/611: where the recognised objects include parts of the human body - H04N 23/67: Focus control based on electronic image sensor signals - G02B 2027/011: {comprising device for correcting geometrical aberrations, distortion} - G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye} - G02F 1/29: for the control of the position or the direction of light beams, i.e. deflection - G06V 40/19: Sensors therefor - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G02F: OPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS ## Abstract An imaging system for producing images for display apparatus. Imaging system includes at least one imaging unit arranged to face real-world scene including camera, optical element including first optical portion and second optical portion having different focal lengths, first focal length of first optical portion is smaller than second focal length of second optical portion, and means for adjusting optical focus; and processor. Processor is configured to obtain gaze direction of user; determine region of interest within real-world scene; and control means for adjusting optical focus of imaging unit, based on focal lengths of first and second optical portions, to capture warped image of real-world scene, the warped image having spatially-uniform angular resolution. ## Claims ### Claim 1 An imaging system for producing images for a display apparatus, the imaging system comprising: at least one imaging unit arranged to face a given real-world scene, a given imaging unit comprising: a camera an optical element that comprises at least a first optical portion and a second optical portion having different focal lengths, wherein a first focal length of the first optical portion is smaller than a second focal length of the second optical portion; and means for adjusting an optical focus of the given imaging unit; and a processor communicably coupled to the at least one imaging unit, wherein the processor is configured to: obtain, from the display apparatus, information indicative of a gaze direction of a user; determine, based on the gaze direction of the user, a region of interest within the given real-world scene; and control the means for adjusting the optical focus of the given imaging unit, based on the focal lengths of the first optical portion and the second optical portion, to capture at least one warped image of the given real-world scene, the at least one warped image having a spatially-uniform angular resolution. ### Claim 2 (depends on 1) The imaging system of claim 1, wherein the imaging system further comprises means for generating a depth or voxel map of the given real-world scene wherein the processor is configured to: determine, based on the depth or voxel map of the given real-world scene, an optical depth of at least one object present in the region of interest; and control the means for adjusting the optical focus of the given imaging unit, based also on the optical depth of at least one object, to capture the at least one warped image of the given real-world scene. ### Claim 3 (depends on 2) The imaging system of claim 2, wherein the at least one object comprises a first object and a second object, a first optical depth of the first object being different from a second optical depth of the second object, wherein the processor is configured to: select a given optical depth that lies between the first optical depth and the second optical depth; and adjust the optical focus of the given imaging unit, based on the given optical depth, to capture the at least one warped image of the given real-world scene. ### Claim 4 (depends on 2) The imaging system of claim 2, wherein the at least one object comprises a first object and a second object, a first optical depth of the first object being different from a second optical depth of the second object, the at least one imaging unit comprising a first imaging unit and a second imaging unit, wherein the processor is configured to adjust an optical focus of the first imaging unit and an optical focus of the second imaging unit, based on the first optical depth and the second optical depth, to capture a first warped image and a second warped image of the given real-world scene, respectively. ### Claim 5 (depends on 1) The imaging system of claim 1, wherein the means for adjusting the optical focus of the given imaging unit comprises at least one first actuator that, in operation, moves the optical element along an optical axis of the camera. ### Claim 6 (depends on 1) The imaging system of claim 1, wherein the means for adjusting the optical focus of the given imaging unit comprises a focusing optical element and at least one second actuator that, in operation, moves the focusing optical element along an optical axis of the camera. ### Claim 7 (depends on 1) The imaging system of claim 1, wherein the optical element and the means for adjusting the optical focus of the given imaging unit are implemented together as a dynamically-controllable optical element, the focal lengths of the first optical portion and the second optical portion of the optical element being dynamically changeable. ### Claim 8 (depends on 1) The imaging system of claim 1, wherein when controlling the means for adjusting the optical focus of the given imaging unit, the processor is configured to adjust, based on the gaze direction of the user, at least one focusing parameter of the optical element. ### Claim 9 (depends on 1) The imaging system of claim 1, wherein the first optical portion substantially surrounds the second optical portion, wherein the second focal length of the second optical portion is more than 1.25 times of the first focal length of the first optical portion. ### Claim 10 (depends on 1) The imaging system of claim 1, wherein the optical element is rotationally asymmetric, the given imaging unit comprising at least one third actuator associated with the optical element, wherein the processor is configured to control the at least one third actuator to adjust a rotational orientation of the optical element according to the gaze direction of the user. ### Claim 11 A method for producing images for a display apparatus, the method being implemented via an imaging system comprising at least one imaging unit arranged to face a given real-world scene, a given imaging unit comprising a camera, an optical element that comprises at least a first optical portion and a second optical portion having different focal lengths, wherein a first focal length of the first optical portion is smaller than a second focal length of the second optical portion, and means for adjusting an optical focus of the given imaging unit, the method comprising: obtaining, from the display apparatus, information indicative of a gaze direction of a user; determining, based on the gaze direction of the user, a region of interest within the given real-world scene; and adjusting an optical focus of the given imaging unit, based on the focal lengths of the first optical portion and the second optical portion, to capture at least one warped image of the given real-world scene, the at least one warped image having a spatially-uniform angular resolution. ### Claim 12 (depends on 11) The method of claim 11, further comprising: generating a depth or voxel map of the given real-world scene; determining, based on the depth or voxel map of the given real-world scene, an optical depth of at least one object present in the region of interest; and adjusting the optical focus of the given imaging unit, based also on the optical depth of at least one object, to capture the at least one warped image of the given real-world scene. ### Claim 13 (depends on 12) The method of claim 12, wherein the at least one object comprises a first object and a second object, a first optical depth of the first object being different from a second optical depth of the second object, wherein the method further comprises: selecting a given optical depth that lies between the first optical depth and the second optical depth; and adjusting the optical focus of the given imaging unit, based on the given optical depth, to capture the at least one warped image of the given real-world scene. ### Claim 14 (depends on 12) The method of claim 12, wherein the at least one object comprises a first object, and a second object, a first optical depth the first object being different from a second optical depth of the second object, the at least one imaging unit comprising a first imaging unit and a second imaging unit, wherein the method further comprises adjusting an optical focus of the first imaging unit and an optical focus of the second imaging unit, based on the first optical depth and the second optical depth, to capture a first warped image and a second warped image of the given real-world scene, respectively. ### Claim 15 (depends on 11) The method of claim 11, wherein the means for adjusting the optical focus of the given imaging unit comprises at least one first actuator associated with the optical element, wherein the step of adjusting the optical focus comprises moving, via the at least one first actuator, the optical element along an optical axis of the camera. ### Claim 16 (depends on 11) The method of claim 11, wherein the means for adjusting the optical focus of the given imaging unit comprises a focusing optical element and at least one second actuator associated therewith, wherein the step of adjusting the optical focus comprises moving, via the at least one second actuator, the focusing optical element along an optical axis of the camera. ### Claim 17 The method of 11, wherein the optical element and the means for adjusting the optical focus of the given imaging unit are implemented together as a dynamically-controllable optical element, wherein the method further comprises dynamically changing the focal lengths of the first optical portion and the second optical portion of the optical element. ### Claim 18 (depends on 11) The method of claim 11, wherein the step of adjusting the optical focus of the given imaging unit comprises adjusting, based on the gaze direction of the user, at least one focusing parameter of the optical element. ### Claim 19 (depends on 11) The method of claim 11, wherein the first optical portion substantially surrounds the second optical portion, wherein the second focal length of the second optical portion is more than 1.25 times of the first focal length of the first optical portion. ### Claim 20 (depends on 11) The method of claim 11, wherein the optical element is rotationally asymmetric, the given imaging unit comprising at least one third actuator associated with the optical element, wherein the method further comprises controlling the at least one third actuator to adjust a rotational orientation of the optical element according to the gaze direction of the user.