# US 2026/0073541 A1 · Image processing method using gaze information and electronic device implementing the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0073541 A1
- Document ID: US20260073541A1
- Status: Published application
- Kind: A1
- Application number: 19/259,865
- Filed: 2025-07-03
- Published: 2026-03-12
- Earliest claimed: 2024-07-03
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Bao HE; Weiming LI; Qiang WANG; Soonyong CHO; Younghun SUNG
- Web page: https://patentgrep.com/patent/US20260073541A1

## CPC

- G06T 7/55: from multiple images
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/73: Deblurring; Sharpening
- G06T 2207/10024: Color image
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method includes obtaining a first image, where the first image includes an RGB image and a first depth image as components of the first image, obtaining at least two image regions by processing the first image through an artificial intelligence (AI) network based on gaze point information, and obtaining a second depth image based on the at least two image regions, where, in the at least two image regions, image qualities of respective image regions are different, and a resolution of the first depth image is lower than a resolution of the second depth image.

## Claims

### Claim 1

A method comprising:
obtaining a first image, wherein the first image comprises an RGB image and a first depth image as components of the first image;
obtaining at least two image regions by processing the first image through an artificial intelligence (AI) network based on gaze point information; and
obtaining a second depth image based on the at least two image regions,
wherein, in the at least two image regions, image qualities of respective image regions are different, and
wherein a resolution of the first depth image is lower than a resolution of the second depth image.

### Claim 2 (depends on 1)

The method of claim 1, wherein the at least two image regions comprise:
a first image region obtained based on an image feature of a first region in the first image centered on a gaze point;
a second image region obtained based on an image feature of a second region in the first image centered on the gaze point, wherein a pixel size of the second region is greater than a pixel size of the first region; and
a third image region obtained based on an image feature of the first image.

### Claim 3 (depends on 1)

The method of claim 1, wherein the obtaining of the at least two image regions comprises:
obtaining a first image region by processing, through a first network, a first region in the first image that is determined based on the gaze point information; and
obtaining at least one second image region by processing, through a second network, a region in the first image other than the first region, and
wherein the second network comprises a partial network structure of the first network.

### Claim 4 (depends on 1)

The method of claim 1, wherein the obtaining of the at least two image regions comprises:
obtaining a shallow feature of the first image;
obtaining at least one image region by performing feature reconstruction on the shallow feature;
obtaining, based on the shallow feature, a deep feature of a first region in the first image that is determined based on the gaze point information; and
obtaining a first image region by performing feature reconstruction on the deep feature.

### Claim 5 (depends on 4)

The method of claim 4, wherein the obtaining of the at least two image regions further comprises:
obtaining a first shallow feature of the first image;
obtaining a third image region by performing feature reconstruction of the first shallow feature;
obtaining, based on the first shallow feature, a second shallow feature of a second region in the first image that is determined based on the gaze point information; and
obtaining a second image region by performing feature reconstruction on the second shallow feature, and
wherein a pixel size of the second region is greater than a pixel size of the first region.

### Claim 6 (depends on 1)

The method of claim 1, wherein the AI network comprises a third network and a fourth network connected in parallel,
wherein the third network comprises:
a first network layer configured to obtain a high frequency feature within the RGB image; and
a second network layer connected to the first network layer and configured to aggregate features output by previous levels,
wherein the fourth network comprises:
a third network layer configured to fuse a depth feature of the first depth image and a high frequency feature output by the third network; and
a fourth network layer connected to the third network layer and configured to aggregate features output by previous levels, and
wherein the obtaining of the at least two image regions comprises:
obtaining, through the third network and based on the RGB image, high frequency features of at least two regions that are determined based on the gaze point information, wherein the high frequency features comprise features representing detail information and/or boundary information;
obtaining, through the fourth network, a first fusion feature corresponding to each of the at least two regions, based on depth features of the at least two regions in the first depth image that are determined based on the high frequency features and the gaze point information; and
obtaining the at least two image regions by performing feature reconstruction for the first fusion feature corresponding to each of the at least two regions.

### Claim 7 (depends on 6)

The method of claim 6, wherein the obtaining of the first fusion feature comprises:
obtaining a second fusion feature by performing feature fusion, based on the high frequency features output by a previous level of the third network and the depth features output by a previous level of the fourth network;
obtaining a third fusion feature by performing multi-scale feature fusion, based on the second fusion feature; and
obtaining the first fusion feature, based on the high frequency features output by the third network of the previous level, the depth features output by the fourth network of the previous level, and the third fusion feature.

### Claim 8 (depends on 7)

The method of claim 7, wherein the obtaining of the second fusion feature comprises:
obtaining a first modulation feature by performing feature modulation based on the high frequency features output by the third network of the previous level and the depth features output by the fourth network of the previous level;
obtaining a second modulation feature by performing feature modulation based on the first modulation feature and the depth features output by the fourth network of the previous level; and
obtaining the second fusion feature based on the depth features output by the fourth network of the previous level and the second modulation feature.

### Claim 9 (depends on 7)

The method of claim 7, wherein the obtaining of the third fusion feature comprises:
obtaining a multi-scale fusion feature by performing multi-scale feature processing based on the second fusion feature;
generating an attention coefficient, based on the second fusion feature;
obtaining a fusion feature related to attention based on the multi-scale fusion feature and the attention coefficient; and
obtaining the third fusion feature based on the fusion feature related to attention and the second fusion feature.

### Claim 10 (depends on 9)

The method of claim 9, wherein the obtaining of the multi-scale fusion feature comprises:
obtaining a feature by performing feature extraction through at least two dilated convolution layers based on the second fusion feature; and
obtaining the multi-scale fusion feature by merging features corresponding to respective dilated convolution layers.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
obtaining a virtual object; and
obtaining a third image comprising the virtual object based on the RGB image and the second depth image.

### Claim 12

An electronic device, comprising:
a memory storing instructions, and;
a processor,
wherein the instructions, when executed by the processor, cause the electronic device to:
obtain a first image comprising an RGB image and a first depth image as components of the first image,
obtain at least two image regions by processing the first image based on gaze point information through an artificial intelligence (AI) network, and
obtain a second depth image based on the at least two image regions,
wherein, in the at least two image regions, image qualities of respective image regions are different, and
wherein a resolution of the first depth image is lower than a resolution of the second depth image.

### Claim 13 (depends on 12)

The electronic device of claim 12, wherein the at least two image regions comprise a first image region obtained based on an image feature of a first region in the first image centered on a gaze point, a second image region obtained based on an image feature of a second region in the first image centered on the gaze point, and a third image region obtained based on an image feature of the first image, and
wherein a pixel size of the second region is greater than a pixel size of the first region.

### Claim 14 (depends on 12)

The electronic device of claim 12, wherein the instructions, when executed by the processor, cause the electronic device to obtain the at least two image regions by:
obtaining, through a first network, a first image region by processing a first region in the first image that is determined based on the gaze point information, and
obtaining, through a second network, at least one other image region by processing a region in the first image other than the first region, and
wherein the second network comprises a partial network structure of the first network.

### Claim 15 (depends on 12)

The electronic device of claim 12, wherein the instructions, when executed by the processor, cause the electronic device to obtain the at least two image regions by:
obtaining a shallow feature of the first image,
obtaining at least one image region by performing feature reconstruction on the shallow feature,
obtaining, based on the shallow feature, a deep feature of a first region in the first image that is determined based on the gaze point information, and
obtaining a first image region by performing feature reconstruction on the deep feature.

### Claim 16 (depends on 15)

The electronic device of claim 15, wherein the instructions, when executed by the processor, cause the electronic device to obtain the at least two image regions by:
obtaining a first shallow feature of the first image,
obtaining a third image region by performing feature reconstruction on the first shallow feature,
obtaining, based on the first shallow feature, a second shallow feature of a second region in the first image that is determined based on the gaze point information, and
obtaining a second image region by performing feature reconstruction on the second shallow feature, and
wherein a pixel size of the second region is greater than a pixel size of the first region.

### Claim 17 (depends on 12)

The electronic device of claim 12, wherein the instructions, when executed by the processor, further cause the electronic device to:
obtain a virtual object, and
obtain a third image comprising the virtual object based on the RGB image and the second depth image.

### Claim 18

A non-transitory, computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to:
obtain a first image, wherein the first image comprises an RGB image and a first depth image as components of the first image;
obtain at least two image regions by processing the first image through an artificial intelligence (AI) network based on gaze point information; and
obtain a second depth image based on the at least two image regions,
wherein, in the at least two image regions, image qualities of respective image regions are different, and
wherein a resolution of the first depth image is lower than a resolution of the second depth image.

### Claim 19 (depends on 18)

The storage medium of claim 18, wherein the at least two image regions comprise:
a first image region obtained based on an image feature of a first region in the first image centered on a gaze point;
a second image region obtained based on an image feature of a second region in the first image centered on the gaze point, wherein a pixel size of the second region is greater than a pixel size of the first region; and
a third image region obtained based on an image feature of the first image.

### Claim 20 (depends on 18)

The storage medium of claim 18, wherein the instructions, when executed by the processor, further cause the processor to obtain the at least two image regions by:
obtaining a first image region by processing, through a first network, a first region in the first image that is determined based on the gaze point information; and
obtaining at least one second image region by processing, through a second network, a region in the first image other than the first region, and
wherein the second network comprises a partial network structure of the first network.