# US 2023/0410718 A1 · Mitigating Artifacts Caused by an Under-Display Light Emitter

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410718 A1
- Document ID: US20230410718A1
- Status: Published application
- Kind: A1
- Application number: 18/184,592
- Filed: 2023-03-15
- Published: 2023-12-21
- Earliest claimed: 2022-06-17
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Jenny Hu; Chaohao Wang; Christopher E Glazowski; Clint M Perlaki; David R Manly; Feng Wen; Graeme M Williams; Hei Kam; Hyun H Boo; Kevin J Choboter; Kyounghwan Kim; Lu Yan; Mahesh B Chappalli; Mark T Winkler; Na Zhu; Peter F Holland; Tong Chen; Warren S Rieutort-Louis; Wenrui Cai; Ximeng Guan; Yingying Tang; Yuchi Che
- Web page: https://patentgrep.com/patent/US20230410718A1

## CPC

- G09G 3/2096: {Details of the interface to the display terminal specific for a flat panel (suitable for both CRT and flat panel G09G5/006; specific for a CRT G09G1/167)}
- G06F 3/042: by opto-electronic means
- G09G 3/3233: with pixel circuitry controlling the current through the light-emitting element
- G09G 3/3406: {Control of illumination source (illumination devices structurally associated with liquid crystal cells G02F1/1336)}
- G06F 3/0412: {Digitisers structurally integrated in a display}
- G09G 2300/0819: used for counteracting undesired variations, e.g. feedback or autozeroing
- G09G 2320/0233: Improving the luminance or brightness uniformity across the screen
- G09G 2320/0285: using tables for spatial correction of display data
- G09G 2320/0295: by monitoring each display pixel
- G09G 2320/041: Temperature compensation
- G09G 2330/06: Handling electromagnetic interferences [EMI], covering emitted as well as received electromagnetic radiation
- G09G 2340/0435: Change or adaptation of the frame rate of the video stream
- G09G 2360/14: Detecting light within display terminals, e.g. using a single or a plurality of photosensors
- G09G 2360/144: the light being ambient light
- G09G 2360/145: the light originating from the display screen
- G09G 2360/16: Calculation or use of calculated indices related to luminance levels in display data
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A light emitter that operates through a display may cause display artifacts, even when the light emitter operates using non-visible wavelengths. Display artifacts caused by a light emitter that operates through a display may be referred to as emitter artifacts. To mitigate emitter artifacts, operating conditions for a display frame may be used to determine an optimal firing time for the light emitter during that display frame. The operating conditions used to determine the optimal firing time may include emitter operating conditions, display content statistics, display brightness, temperature, and refresh rate. Operating conditions from one or more previous frames may be stored in a frame buffer and may be used to help determine the optimal firing time for the light emitter during a display frame. Pixel values for the display may be modified to mitigate emitter artifacts.

## Claims

### Claim 1

An electronic device, comprising:
a display comprising pixels;
a sensor comprising a light emitter that emits light through the display; and
control circuitry configured to:
based on operating conditions for a given display frame, determine an optimal firing time for the light emitter in the sensor; and
control the light emitter to emit light at the optimal firing time during the given display frame.

### Claim 2 (depends on 1)

The electronic device defined in claim 1, wherein the operating conditions for the given display frame comprise operating conditions for the light emitter.

### Claim 3 (depends on 2)

The electronic device defined in claim 2, wherein the operating conditions for the light emitter comprise an operating condition selected from the group consisting of: a wavelength for the light emitter, an irradiation power for the light emitter, a pulse duration for the light emitter, and a firing frequency for the light emitter.

### Claim 4 (depends on 1)

The electronic device defined in claim 1, wherein the operating conditions for the given display frame comprise statistics for a subset of the pixels.

### Claim 5 (depends on 4)

The electronic device defined in claim 4, wherein the statistics for the subset of the pixels comprise color and brightness information for the subset of the pixels.

### Claim 6 (depends on 5)

The electronic device defined in claim 5, wherein the light from the light emitter passes through the subset of the pixels.

### Claim 7 (depends on 5)

The electronic device defined in claim 5, wherein the control circuitry is configured to, for each pixel in the subset of the pixels, determine an emitter artifact profile for that pixel using a look-up table.

### Claim 8 (depends on 7)

The electronic device defined in claim 7, wherein the control circuitry is configured to use a weighted average of the emitter artifact profiles for the subset of the pixels to obtain a single representative emitter artifact profile for the subset of the pixels and wherein determining the optimal firing time for the light emitter in the sensor comprises determine the optimal firing time for the light emitter in the sensor based on the single representative emitter artifact profile.

### Claim 9 (depends on 5)

The electronic device defined in claim 5, wherein the control circuitry is configured to, for each pixel in the subset of the pixels, determine an emitter artifact profile for that pixel using a predictive model.

### Claim 10 (depends on 9)

The electronic device defined in claim 9, wherein the control circuitry is configured to use a weighted average of the emitter artifact profiles for the subset of the pixels to obtain a single representative emitter artifact profile for the subset of the pixels and wherein determining the optimal firing time for the light emitter in the sensor comprises determine the optimal firing time for the light emitter in the sensor based on the single representative emitter artifact profile.

### Claim 11 (depends on 1)

The electronic device defined in claim 1, wherein the operating conditions for the given display frame comprise a display brightness.

### Claim 12 (depends on 1)

The electronic device defined in claim 1, further comprising:
a temperature sensor, wherein the operating conditions for the given display frame comprise a temperature from the temperature sensor.

### Claim 13 (depends on 1)

The electronic device defined in claim 1, wherein the operating conditions for the given display frame comprise a refresh rate for the display.

### Claim 14 (depends on 1)

The electronic device defined in claim 1, wherein determining the optimal firing time for the light emitter in the sensor based on the operating conditions for the given display frame comprises determining the optimal firing time for the light emitter in the sensor based on the operating conditions for the given display frame and operating conditions for at least one previous display frame.

### Claim 15 (depends on 14)

The electronic device defined in claim 14, further comprising:
a frame buffer configured to store the operating conditions for at least one previous display frame.

### Claim 16 (depends on 1)

The electronic device defined in claim 1, wherein the control circuitry is configured to:
determine a difference between an operating condition for the given display frame and the operating condition for a previous display frame; and
in response to the difference being less than a threshold, control the light emitter to emit light at a previously determined optimal firing time during the given display frame.

### Claim 17 (depends on 1)

The electronic device defined in claim 1, wherein the control circuitry is configured to:
determine a difference between an operating condition for the given display frame and the operating condition for a previous display frame, wherein determining the optimal firing time for the light emitter in the sensor based on the operating conditions for the given display frame comprises determining the optimal firing time for the light emitter in the sensor based on the operating conditions for the given display frame in response to the difference being greater than a threshold.

### Claim 18 (depends on 1)

The electronic device defined in claim 1, wherein the light emitter is an infrared light emitter that emits infrared light.

### Claim 19

An electronic device, comprising:
a display comprising pixels;
a proximity sensor comprising a light emitter that emits light through the display; and
control circuitry configured to, based on operating conditions for a given display frame, modify pixel data for the given display frame to mitigate artifacts caused by the light from the light emitter passing through the display.

### Claim 20

An electronic device, comprising:
a display comprising pixels;
a proximity sensor comprising a light emitter that emits light through the display; and
control circuitry configured to:
for each pixel in a subset of the pixels, determine an emitter artifact profile for that pixel using at least a brightness of that pixel for a given display frame and a look-up table;
spatially weight the emitter artifact profiles for the subset of the pixels to obtain a single representative emitter artifact profile for the subset of the pixels;
use the single representative emitter artifact profile to determine an optimal firing time for the light emitter in the proximity sensor; and
control the light emitter to emit light at the optimal firing time during the given display frame.
