US 11,445,170 B2Granted

Eyeball camera system and methods for display system calibration

AssigneeMagic Leap, Inc.

InventorsZhiheng Jia, Jeffrey Todd Daiker, Timothy Alan Ellis, Xiao Li, Jeremy A. Grata

Life of this patent

Estimating the term…

Loading patent…

Abstract

Embodiments provide a camera design (e.g., an eyeball camera) that mimics a human eye in geometry, optical performance and/or motion. The eyeball camera adopts the same cornea and pupil geometry from the human eye, and has the iris and pupil configured with multiple texture, color or diameter options. The resolution of the eyeball camera is designed to match the acuity of typical 20/20 human vision, and focus is adjusted from 0 to 4 diopters. A pair of eyeball cameras are mounted independently on two hexapods to simulate the human eye gaze and vergence. The perceived virtual and real world are calibrated and evaluated based on eye conditions like pupil location and gaze using the eyeball cameras. The eyeball camera serves as a bridge to combine the data from spatial computing like eye tracking, 3D geometry of the digital world, display color accuracy/uniformity, and display optical quality (sharpness, contrast, etc.).

Claims24

Dependent claims

A system for applying a correction to a virtual image display system, the system comprising:

a display device configured to display a virtual image;
an image capture device positioned in front of the display device, wherein the image capture device has one or more properties similar to a human eye;
and
a processor coupled to the display device and the image capture device to receive image data from the image capture device, wherein the processor is programmed to:
receive a first image of a scene;
determine a first set of visual properties associated with the first image;
receive a second image of the scene captured with the image capture device;
determine a second set of visual properties associated with the second image, wherein the first set of visual properties and the second set of visual properties include at least one of a spatial positioning, a color or a geometry attribute;
identify a discrepancy between at least one of the first set of visual properties of the first image and at least one of the second set of visual properties of the second image corresponding to he at least one of the first set of visual properties of the first image;
determine a correction to apply to the virtual image display system to at least partially correct for the discrepancy using the one or more properties of the image capture device, wherein the correction matches the spatial positioning, the color or the geometry attribute associated with the first image to that of the second image;
and
apply the correction to the virtual image display system based on the one or more properties of the image capture device.

9 more elements

A method for applying a correction to a virtual image display system, the method comprising:

receiving, by a processor, a first image of a scene from a real-world image capture device;
determining, by the processor, a first set of visual properties associated with the first image;
receiving, by the processor from an image capture device, a second image of the scene captured with the image capture device having one or more properties similar to a human eye, wherein the image capture device is positioned in front of a display device;
determining, by the processor, a second set of visual properties associated with the second image, wherein the first set of visual properties and the second set of visual properties include at least one of a spatial positioning, a color or a geometry attribute;
identifying, by the processor, a discrepancy between at least one of the first set of visual properties of the first image and at let one of the second set of visual properties of the second image corresponding to the at least one of first set of visual properties of the first image;
determining, by the processor, a correction to apply to the virtual image display system to at least partially correct for the discrepancy using the one or more properties of the image capture device, wherein the correction matches the spatial positioning, the color or the geometry attribute associated with the first image to that of the second image;
and
applying, by the processor, the correction to the virtual image display system based on the one or more properties of the image capture device.

5 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/926,570Filed 10 Jul 2020 · us provisional application
Priority: 12 Jul 2019
Published: US 2021/0014477 A114 Jan 2021
Granted: 13 Sep 2022
Term ends: Not estimatedThe source records don’t support an estimate.

Classification15 codes

G06T 7/85
Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration: stereo camera calibration
H04N 17/004
Diagnosis, testing or measuring for television systems or their details: for digital television systems
G02B 27/0093
Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00: with means for monitoring data relating to the…
G02B 27/01
Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00: head-up displays

Family

Application62/873,641Us provisional application · filed 12 Jul 2019
US 2021/0014477 A1Pre-grant publication · published 14 Jan 2021
US 11,445,170 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key