# US 2018/0324332 A1 · Display apparatus and method using image renderers and optical combiners

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0324332 A1
- Document ID: US20180324332A1
- Status: Published application
- Kind: A1
- Application number: 15/584,497
- Filed: 2017-05-02
- Published: 2018-11-08
- Earliest claimed: 2017-05-02
- Assignees: Varjo Technologies Oy
- Applicants: Varjo Technologies Oy
- Inventors: Urho Konttori; Oiva Arvo Oskari Sahlsten; Kseniia Duginova
- Web page: https://patentgrep.com/patent/US20180324332A1

## CPC

- H04N 23/811: {by dust removal, e.g. from surfaces of the image sensor or processing of the image signal output by the electronic image sensor}
- G02B 3/0037: {characterized by the distribution or form of lenses}
- G02B 6/3504: {Rotating, tilting or pivoting the waveguides, or with the waveguides describing a curved path (rotary joint G02B6/3628)}
- G02B 27/0172: {characterised by optical features}
- G06T 5/73: Deblurring; Sharpening
- G06T 15/503: {Blending, e.g. for anti-aliasing}
- H04N 13/156: Mixing image signals
- H04N 13/383: for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
- H04N 23/67: Focus control based on electronic image sensor signals
- G02B 5/20: Filters (polarising elements G02B5/30)
- G02B 2027/0123: {comprising devices increasing the field of view}
- G02B 2027/013: {comprising a combiner of particular shape, e.g. curvature}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0147: {comprising a device modifying the resolution of the displayed image}
- G02B 2027/0161: {characterised by the relative positioning of the constitutive elements}
- G02B 2027/0174: {holographic}
- G02B 2027/0178: {Eyeglass type (eyeglass details G02C)}
- G02B 2027/0187: {slaved to motion of at least a part of the body of the user, e.g. head, eye}
- G06T 7/73: using feature-based methods
- H04N 13/344: with head-mounted left-right displays
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A display apparatus, and a method of displaying via the display apparatus. The display apparatus includes at least one context image renderer for rendering a context image, at least one focus image renderer for rendering a focus image, at least one first optical combiner for combining the projection of the rendered context image with the projection of the rendered focus image to form a combined projection, and at least one second optical combiner for combining the combined projection with a projection of a real world image. An angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees. An angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees.

## Claims

### Claim 1

A display apparatus comprising:
at least one context image renderer for rendering a context image, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees;
at least one focus image renderer for rendering a focus image, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees;
at least one first optical combiner for combining the projection of the rendered context image with the projection of the rendered focus image to form a combined projection; and
at least one second optical combiner for combining the combined projection with a projection of a real world image.

### Claim 2 (depends on 1)

The display apparatus of claim 1, wherein the at least one second optical combiner is switchable to different levels of transparency.

### Claim 3 (depends on 1)

The display apparatus of claim 1, further comprising a varifocal element for adjusting a focus of the projection of the rendered focus image.

### Claim 4 (depends on 1)

The display apparatus of claim 1, wherein the at least one second optical combiner has a first surface facing the real world and a second surface substantially opposite to the first surface, and wherein the first surface is substantially reflective, while the second surface is substantially anti-reflective.

### Claim 5 (depends on 1)

The display apparatus of claim 1, wherein the at least one second optical combiner is curved in shape.

### Claim 6 (depends on 1)

The display apparatus of claim 1, wherein the at least one context image renderer and/or the at least one focus image renderer are implemented by way of at least one projector and at least one projection screen, wherein the at least one projection screen is curved in shape.

### Claim 7 (depends on 1)

The display apparatus of claim 1, further comprising:
means for detecting a gaze direction; and
a processor coupled in communication with the at least one first optical combiner, the at least one second optical combiner and the means for detecting the gaze direction, wherein the processor is configured to:
(a) obtain an input image, and use the detected gaze direction to determine a region of visual accuracy of the input image;
(b) process the input image to generate the context image and the focus image, the context image having a first resolution and the focus image having a second resolution, wherein:
(i) a region of the context image that substantially corresponds to the region of visual accuracy of the input image is masked,
(ii) the focus image substantially corresponds to the region of visual accuracy of the input image, and
(iii) the second resolution is higher than the first resolution; and
(c) render the context image at the at least one context image renderer and the focus image at the at least one focus image renderer substantially simultaneously, whilst controlling the at least one first optical combiner to combine the projection of the rendered context image with the projection of the rendered focus image in a manner that the projection of the rendered focus image substantially overlaps the projection of the masked region of the rendered context image.

### Claim 8 (depends on 7)

The display apparatus of claim 7, further comprising at least one camera coupled in communication with the processor, wherein the at least one camera is positioned in a proximity of a first surface of the at least one second optical combiner facing the real world, and is arranged for capturing the projection of the real world image reflected towards the at least one camera from the first surface.

### Claim 9 (depends on 8)

The display apparatus of claim 8, wherein the processor is operable to combine the at least one image of the real world with the combined projection to reduce vergence-accommodation conflict.

### Claim 10

A method of displaying, via a display apparatus comprising at least one context image renderer, at least one focus image renderer, at least one first optical combiner and at least one second optical combiner, the method comprising:
(i) rendering a context image at the at least one context image renderer, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees;
(ii) rendering a focus image at the at least one focus image renderer, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees; and
(iii) using the at least one first optical combiner to combine the projection of the rendered context image with the projection of the rendered focus image to form a combined projection; and
(iv) using the at least one second optical combiner to combine the combined projection with a projection of a real world image.

### Claim 11 (depends on 10)

The method of claim 10, further comprising switching the at least one second optical combiner from a first level of transparency to a second level of transparency.

### Claim 12 (depends on 10)

The method of claim 10, further comprising using a varifocal element of the display apparatus to adjust a focus of the projection of the rendered focus image.

### Claim 13 (depends on 10)

The method of claim 10, wherein the at least one second optical combiner has a first surface facing the real world and a second surface substantially opposite to the first surface, and wherein the first surface is substantially reflective, while the second surface is substantially anti-reflective.

### Claim 14 (depends on 10)

The method of claim 10, wherein the at least one second optical combiner is curved in shape.

### Claim 15 (depends on 10)

The method of claim 10, wherein the at least one context image renderer and/or the at least one focus image renderer are implemented by way of at least one projector and at least one projection screen, wherein the at least one projection screen is curved in shape.

### Claim 16 (depends on 10)

The method of claim 10, wherein the display apparatus further comprises means for detecting a gaze direction, and wherein the method further comprises:
(iv) detecting a gaze direction, and using the detected gaze direction to determine a region of visual accuracy of an input image;
(v) processing the input image to generate the context image and the focus image, the context image having a first resolution and the focus image having a second resolution, the second resolution being higher than the first resolution, wherein the processing comprises:
masking a region of the context image that substantially corresponds to the region of visual accuracy of the input image; and
generating the focus image to substantially correspond to the region of visual accuracy of the input image; and
(vi) controlling the at least one first optical combiner to combine the projection of the rendered context image with the projection of the rendered focus image in a manner that the projection of the rendered focus image substantially overlaps the projection of the masked region of the rendered context image,
wherein (i), (ii) and (vi) are performed substantially simultaneously.

### Claim 17 (depends on 16)

The method of claim 16, wherein the display apparatus further comprises at least one camera coupled in communication with the processor, wherein the at least one camera is positioned in a proximity of a first surface of the at least one second optical combiner facing the real world, and is arranged for capturing the projection of the real world image reflected towards the at least one camera from the first surface.

### Claim 18 (depends on 17)

The method of claim 17, wherein the method further comprises combining the at least one image of the real world with the combined projection to reduce vergence-accommodation conflict.
