# US 2016/0260258 A1 · Apparatuses, methods and systems coupling visual accommodation and visual convergence to the same plane at any depth of an object of interest

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0260258 A1
- Document ID: US20160260258A1
- Status: Published application
- Kind: A1
- Application number: 14/757,464
- Filed: 2015-12-23
- Published: 2016-09-08
- Earliest claimed: 2014-12-23
- Assignees: Meta Company
- Applicants: Meta Company
- Inventors: Raymond Chun Hing Lo; Stefano Baldassi
- Web page: https://patentgrep.com/patent/US20160260258A1

## CPC

- G02B 30/10: using integral imaging methods
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G02B 3/14: of variable focal length
- G02B 27/0075: {with means for altering, e.g. increasing, the depth of field or depth of focus}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G02B 27/0172: {characterised by optical features}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/1423: {controlling a plurality of local displays, e.g. CRT and flat panel display}
- G06F 3/147: using display panels
- G06T 7/00: Image analysis
- G06T 15/00: Three-dimensional [3D] image rendering
- G06T 15/20: Perspective computation
- G09G 3/003: {to produce spatial visual effects}
- G09G 3/3208: organic, e.g. using organic light-emitting diodes [OLED]
- G09G 3/3611: {Control of matrices with row and column drivers}
- G09G 5/363: {Graphics controllers}
- H04N 13/00: Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N 13/344: with head-mounted left-right displays
- H04N 13/383: for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
- H04N 13/388: Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume
- G02B 2027/0127: {comprising devices increasing the depth of field}
- G02B 2027/0134: {of stereoscopic type}
- G02B 2027/0185: {Displaying image at variable distance}
- G09G 2300/023: Display panel composed of stacked panels
- G09G 2320/0233: Improving the luminance or brightness uniformity across the screen
- G09G 2320/0626: for control of overall brightness
- G09G 2340/04: Changes in size, position or resolution of an image
- G09G 2340/045: Zooming at least part of an image, i.e. enlarging it or shrinking it
- G09G 2340/0464: Positioning
- G09G 2340/14: Solving problems related to the presentation of information to be displayed
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Aspects of the disclosed apparatuses, methods and systems provide three dimensional gradient and dynamic light fields for display in 3D technologies, in particular 3D augmented reality (AR) devices, by coupling visual accommodation and visual convergence to the same plane at any depth of an object of interest in real time.

## Claims

### Claim 1

A vision system comprising:
a first display;
a second display;
an optical element arranged at a first distance from the first display and at a second distance from the second display to reflect project light emitted from the first display in a first focal plane and to project light emitted from the second display in a second focal plane; and
a processing device configured to control the relative intensity (RI) of one or more pixels of the first display and the second display to generate a perceived three dimensional light field.

### Claim 2 (depends on 1)

The system of claim 1 wherein the first focal plane is a far plane (FP) of focus to a viewer of the system and the second focal plane is a near plane (NP) of focus to a viewer of the system.

### Claim 3 (depends on 2)

The system of claim 2 wherein RI=(INP−IFP)/(INP+IFP)
where RI is the relative intensity calculated at each pixel of an image and normalized, INP is the intensity of the image provided in the NP, IFP is the intensity of the image provided in the FP.

### Claim 4 (depends on 1)

The system of claim 1 wherein the first display is a liquid crystal display and the second display is a transparent organic light emitting diode display and the optical element has a single focus.

### Claim 5 (depends on 1)

The system of claim 1 further comprising a third display arranged at a third distance from the optical element that is closer than the first display and farther than the second display, the third display arranged to project an image in a mid-plane of focus of the viewer of the system.

### Claim 6

A vision system comprising:
a display including at least a first portion and a second portion;
a multifocal optical element including a first focus arranged to project light emitted from the first portion of the display in a first focal plane and a second focus arranged to project light emitted from the second portion of the display in a second focal plane; and
a processing device configured to control the relative intensity (RI) of one or more pixels of the first portion of the display and the second portion of the display to generate a perceptual three dimensional light field by.

### Claim 7 (depends on 6)

The system of claim 6 wherein the first focal plane is a far plane (FP) of focus to a viewer of the system and the second focal plane is a near plane (NP) of focus to a viewer of the system.

### Claim 8 (depends on 7)

The system claim 7 wherein RI=(INP−IFP)/(INP+IFP)
where RI is the relative intensity calculated at each pixel of an image and normalized, INP is the intensity of the image provided in the NP, IFP is the intensity of the image provided in the FP.

### Claim 9 (depends on 6)

The system of claim 6 wherein the display includes a third portion and the multifocal optical element include a third focus arranged to project an image from the third portion in a mid-plane of focus of the viewer of the system.

### Claim 10

A method of generating a gradient light field for a vision system, the method comprising:
receiving image data by at least one component of the vision system;
determining a perceived depth within the gradient light field formed by at least two focal planes by the component for a pixel of the image data to be presented to the viewer;
determining a relative intensity (RI) the pixel of the image data in each focal plane, the RI corresponding to the determined perceived depth of the pixel in the light field; and
controlling a pixel of a corresponding light source or portion of a light source to illuminate at an intensity corresponding to the determined RI.

### Claim 11 (depends on 10)

The method of claim 10 wherein RI=(INP−IFP)/(INP+IFP)
where RI is the relative intensity calculated at each pixel of an image and normalized, INP is the intensity of the image provided in a near plane of focus of the viewer, IFP is the intensity of the image provided in a far plane of focus of the viewer.

### Claim 12

A vision system comprising:
an eye tracking device configured to determine a position of the eyes of a viewer of the system;
a multifocal optical element; and
a processing device configured to determine the convergence point in a three dimensional space of the viewer of the vision system, to adjust a focal plane of the vision system based on the determined convergence point, and to render depth dependent disparity and blur by the vision system for a displayed image based on the adjusted focal plane.

### Claim 13

A method of generating a perceptual dynamic light field to a viewer of a vision system, the method comprising:
determining the convergence point in three dimensional space of the viewer of the vision system;
adjusting the focal plane of the vision system based on the determined convergence point; and
rendering depth dependent disparity and blur by the vision system for a displayed image based on the adjusted focal plane.

### Claim 14 (depends on 13)

The method of claim 13 further comprising:
directing light from a source at the eyes of the viewer;
sensing the reflectance of the light source from the eyes; and
determining the eye position of each eye of the viewer,
wherein the convergence point is determined based on the determined eye positions of the viewer.
