# US 2021/0241424 A1 · Display apparatus and method of compensating for visual artifacts

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241424 A1
- Document ID: US20210241424A1
- Status: Published application
- Kind: A1
- Application number: 16/778,760
- Filed: 2020-01-31
- Published: 2021-08-05
- Earliest claimed: 2020-01-31
- Assignees: Varjo Technologies Oy
- Applicants: Varjo Technologies Oy
- Inventors: Ari Antti Peuhkurinen
- Web page: https://patentgrep.com/patent/US20210241424A1

## CPC

- G02B 27/0172: {characterised by optical features}
- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 3/00: Geometric image transformations in the plane of the image
- G09G 5/10: Intensity circuits
- G02B 2027/012: {comprising devices for attenuating parasitic image effects}
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G09G 2320/0261: in the context of movement of objects on the screen or movement of the observer relative to the screen
- G09G 2320/0686: with two or more screen areas displaying information with different brightness or colours
- G09G 2320/08: Arrangements within a display terminal for setting, manually or automatically, display parameters of the display terminal
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

A display apparatus including first display or projector for displaying first images for first eye; second display or projector for displaying second images for second eye; first portion and second portion arranged to face first and second eyes; means for tracking poses of first and second eyes relative to first and second optical axes, respectively; and processor. Processor or external processor is configured to: obtain given pose of given eye relative to given optical axis; generate information pertaining to given visual artifact that is formed over given image at image plane when given image is displayed; determine artifact-superposing portion of given image; and process given image based on generated information and artifact-superposing portion, to generate given artifact-compensated image. Processor displays given artifact-compensated image via given display or projector.

## Claims

### Claim 1

A display apparatus comprising:
at least one first display or projector that, in operation, displays first images to be presented to a first eye of a user;
at least one second display or projector that, in operation, displays second images to be presented to a second eye of the user;
a first portion and a second portion that are arranged to face the first eye and the second eye, respectively, the first portion and the second portion having a first optical axis and a second optical axis, respectively;
means for tracking a pose of the first eye and a pose of the second eye relative to the first optical axis and the second optical axis, respectively; and
a processor coupled to the at least one first display or projector, the at least one second display or projector and said means, wherein the processor or at least one external processor communicably coupled to the processor is configured to:
obtain a given pose of a given eye relative to a given optical axis of a given portion of the display apparatus;
generate, based on the given pose of the given eye and a given image to be presented to the given eye, information pertaining to a given visual artifact that is formed over the given image at an image plane when the given image is displayed at a given display or projector;
determine, based on the given pose of the given eye, an artifact-superposing portion of the given image on which the given visual artifact superimposes at the image plane when the given image is displayed at the given display or projector; and
process the given image based on the generated information and the determined artifact-superposing portion, to generate a given artifact-compensated image, by adjusting an intensity of pixels within the artifact-superposing portion based on an intensity of pixels within the given visual artifact,
wherein the processor is configured to display the given artifact-compensated image via the given display or projector.

### Claim 2 (depends on 1)

The display apparatus of claim 1, wherein, when obtaining the given pose of the given eye, the processor or the at least one external processor is configured to process eye-tracking data obtained from said means to determine the given pose of the given eye relative to the given optical axis of the given portion of the display apparatus.

### Claim 3 (depends on 1)

The display apparatus of claim 1, wherein, when obtaining the given pose of the given eye, the processor or the at least one external processor is configured to:
process eye-tracking data obtained from said means to determine a plurality of poses of the given eye relative to the given optical axis of the given portion of the display apparatus; and
determine the given pose of the given eye by calculating a mean or median of the plurality of poses of the given eye.

### Claim 4 (depends on 1)

The display apparatus of claim 1, wherein the information pertaining to the given visual artifact comprises an artifact image that is representative of the given visual artifact.

### Claim 5

(canceled)

### Claim 6 (depends on 1)

The display apparatus of claim 1, further comprising means for tracking a gaze direction of the first eye and a gaze direction of the second eye, said means being coupled to the processor, wherein the processor or the at least one external processor is configured to obtain a given gaze direction of the given eye and to determine, based on the given gaze direction of the given eye, a region of interest within the given image,
wherein, when processing the given image, the processor or the at least one external processor is configured to:
detect whether or not the artifact-superposing portion of the given image overlaps with the region of interest within the given image; and
adjust the intensity of the pixels within the artifact-superposing portion when it is detected that the artifact-superposing portion of the given image overlaps with the region of interest within the given image.

### Claim 7 (depends on 4)

The display apparatus of claim 4, wherein, when generating the information pertaining to the given visual artifact, the processor or the at least one external processor is configured to:
determine, based on the given pose of the given eye, a deforming transformation to be applied to the given image; and
apply the deforming transformation to the given image to generate the artifact image.

### Claim 8 (depends on 7)

The display apparatus of claim 7, wherein, when generating the information pertaining to the given visual artifact, the processor or the at least one external processor is configured to bin pixels of the given image prior to applying the deforming transformation, wherein a resolution of the artifact image is lower than a resolution of the given image.

### Claim 9 (depends on 7)

The display apparatus of claim 7, wherein, when generating the information pertaining to the given visual artifact, the processor or the at least one external processor is configured to determine an artifact-causing portion of the given image that causes formation of the given visual artifact, wherein the deforming transformation is to be applied to the artifact-causing portion of the given image.

### Claim 10 (depends on 1)

The display apparatus of claim 1, wherein the processor or the at least one external processor is configured to:
generate, based on the given pose of the given eye and the given image, information pertaining to at least one other visual artifact that is formed outside a periphery of the given image at the image plane when the given image is displayed at the given display or projector;
determine at least one other artifact-causing portion of the given image that causes formation of the at least one other visual artifact; and
adjust, based on the information pertaining to the at least one other visual artifact, an intensity of individual pixels within the at least one other artifact-causing portion of the given image.

### Claim 11

A method of compensating for visual artifacts in a display apparatus, the method comprising:
obtaining a given pose of a given eye of a user relative to a given optical axis of a given portion of the display apparatus;
generating, based on the given pose of the given eye and a given image to be presented to the given eye, information pertaining to a given visual artifact that is formed over the given image at an image plane when the given image is displayed at a given display or projector of the display apparatus;
determining, based on the given pose of the given eye, an artifact-superposing portion of the given image on which the given visual artifact superimposes at the image plane when the given image is displayed at the given display or projector;
processing the given image based on the generated information and the determined artifact-superposing portion, to generate a given artifact-compensated image by adjusting an intensity of pixels within the artifact-superposing portion based on an intensity of pixels within the given visual artifact; and
displaying the given artifact-compensated image via the given display or projector.

### Claim 12 (depends on 11)

The method of claim 11, wherein the step of obtaining the given pose of the given eye comprises processing eye-tracking data to determine the given pose of the given eye relative to the given optical axis of the given portion of the display apparatus.

### Claim 13 (depends on 11)

The method of claim 11, wherein the step of obtaining the given pose of the given eye comprises:
processing eye-tracking data to determine a plurality of poses of the given eye relative to the given optical axis of the given portion of the display apparatus; and
determining the given pose of the given eye by calculating a mean or median of the plurality of poses of the given eye.

### Claim 14 (depends on 11)

The method of claim 11, wherein the information pertaining to the given visual artifact comprises an artifact image that is representative of the given visual artifact.

### Claim 15

(canceled)

### Claim 16 (depends on 11)

The method of claim 11, further comprising:
obtaining a given gaze direction of the given eye; and
determining, based on the given gaze direction of the given eye, a region of interest within the given image,
wherein the step of processing the given image comprises:
detecting whether or not the artifact-superposing portion of the given image overlaps with the region of interest within the given image; and
adjusting the intensity of the pixels within the artifact-superposing portion when it is detected that the artifact-superposing portion of the given image overlaps with the region of interest within the given image.

### Claim 17 (depends on 14)

The method of claim 14, wherein the step of generating the information pertaining to the given visual artifact comprises:
determining, based on the given pose of the given eye, a deforming transformation to be applied to the given image; and
applying the deforming transformation to the given image to generate the artifact image.

### Claim 18 (depends on 17)

The method of claim 17, wherein the step of generating the information pertaining to the given visual artifact comprises binning pixels of the given image prior to applying the deforming transformation, wherein a resolution of the artifact image is lower than a resolution of the given image.

### Claim 19 (depends on 17)

The method of claim 17, wherein the step of generating the information pertaining to the given visual artifact comprises:
determining an artifact-causing portion of the given image that causes formation of the given visual artifact; and
applying the deforming transformation to the artifact-causing portion of the given image.

### Claim 20 (depends on 11)

The method of claim 11, further comprising:
generating, based on the given pose of the given eye and the given image, information pertaining to at least one other visual artifact that is formed outside a periphery of the given image at the image plane when the given image is displayed at the given display or projector;
determining at least one other artifact-causing portion of the given image that causes formation of the at least one other visual artifact; and
adjusting, based on the information pertaining to the at least one other visual artifact, an intensity of individual pixels within the at least one other artifact-causing portion of the given image.
