# US 2024/0005464 A1 · Reflection removal from an image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005464 A1
- Document ID: US20240005464A1
- Status: Published application
- Kind: A1
- Application number: 18/467,842
- Filed: 2023-09-15
- Published: 2024-01-04
- Earliest claimed: 2021-03-16
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Renat Ahmirovich KHIZBULLIN; Pavel Aleksandrovich OSTYAKOV; Stamatis LEFKIMMIATIS
- Web page: https://patentgrep.com/patent/US20240005464A1

## CPC

- G06T 5/20: using local operators
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0475: Generative networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/094: Adversarial learning
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 5/77: Retouching; Inpainting; Scratch removal
- G06T 7/11: Region-based segmentation
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model
- G06N 3/047: Probabilistic or stochastic networks
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30196: Human being; Person
- G06T 2207/30201: Face
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The technology of this application relates to a method for removing reflections from an image. The method detects one or more reflection areas in the image, wherein each reflection area includes a reflection. Further, the method extracts the one or more reflection areas from the image, and removes the reflection from each of the extracted reflection areas.

## Claims

### Claim 1

A method for removing reflections from an image, the method comprising:
detecting one or more reflection areas in the image, wherein each reflection area, from the one or more reflection areas, includes a reflection;
extracting the one or more reflection areas from the image; and
removing the reflection from each of the extracted reflection areas.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
after removing the reflection from each of the extracted reflection areas,
reinserting the extracted reflection areas without the reflection into the image to replace, respectively, the reflection areas with the reflection.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
detecting the one or more reflection areas using a first trained model; and/or
removing the reflection from each of the extracted reflection areas using a second trained model.

### Claim 4 (depends on 3)

The method of claim 3, wherein the first trained model includes a first convolutional neural network (CNN).

### Claim 5 (depends on 4)

The method of claim 4, wherein the first CNN includes a semantic segmentation CNN configured to perform a semantic segmentation of the image.

### Claim 6 (depends on 5)

The method of claim 5, wherein the one or more reflection areas are detected using a semantic mask.

### Claim 7 (depends on 3)

The method of claim 3, wherein the second trained model includes a generative adversarial network (GAN).

### Claim 8 (depends on 7)

The method of claim 7, wherein the GAN includes a conditional GAN.

### Claim 9 (depends on 3)

The method of claim 3, wherein the second trained model comprises includes a second convolutional neural network.

### Claim 10 (depends on 1)

The method of claim 1, wherein
the image is a photograph of a person wearing eyeglasses, and
detecting the one or more reflection areas comprises:
detecting a face of the person in the image;
detecting the eyeglasses in the image based on the detected face; and
detecting the one or more reflection areas located within the eyeglasses detected in the image.

### Claim 11 (depends on 5)

The method of claim 5, wherein eyeglasses are detected in the image using the semantic segmentation CNN.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
segmenting the eyeglasses detected in the image using the semantic segmentation CNN, into eyeglass segments;
extracting the eyeglass segments from the image;
detecting the one or more reflection areas located within the eyeglasses by removing the eyeglass segments without reflection from the extracted eyeglasses segments; and
removing the reflection from each of the extracted eyeglasses segments with reflection.

### Claim 13

A device configured to remove reflections from an image, the device comprising:
a processor; and
a memory configured to store computer readable instructions that, when executed by the processor, cause the device to:
detect one or more reflection areas in the image, wherein each reflection area, from the one or more reflection areas, includes at least one reflection,
extract the one or more reflection areas from the image, and
remove the at least one reflection from each of the extracted reflection areas.

### Claim 14 (depends on 1)

A computer program comprising program code for performing the method according to claim 1.

### Claim 15

A non-transitory computer readable storage medium configured to store computer readable instructions that, when executed by a processor, cause the processor to provide execution comprising:
detecting one or more reflection areas in the image;
extracting the one or more reflection areas from the image; and
removing the reflection from each of the extracted reflection areas.

### Claim 16 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the processor is further caused to provide execution comprising:
reinserting the extracted reflection areas without the reflection into the image.

### Claim 17 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the processor is further caused to provide execution comprising:
detecting the one or more reflection areas using a first trained model; and/or
removing the reflection from each of the extracted reflection areas using a second trained model.

### Claim 18 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the first trained model includes a first convolutional neural network (CNN).

### Claim 19 (depends on 18)

The non-transitory computer readable storage medium of claim 18, wherein the first CNN includes a semantic segmentation CNN configured to perform a semantic segmentation of the image.

### Claim 20 (depends on 19)

The non-transitory computer readable storage medium of claim 19, wherein the one or more reflection areas are detected using a semantic mask.
