# US 2025/0157205 A1 · Method and apparatus with image enhancement using base image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0157205 A1
- Document ID: US20250157205A1
- Status: Published application
- Kind: A1
- Application number: 18/934,024
- Filed: 2024-10-31
- Published: 2025-05-15
- Earliest claimed: 2023-11-09
- Assignees: Samsung Electronics Co., Ltd.; Postech Research and Business Development Foundation
- Applicants: Samsung Electronics Co., Ltd.; Postech Research and Business Development Foundation
- Inventors: Jae seok CHOI; Manjin KIM; SangHyun KIM; Woohyeok KIM; Insoo KIM; Min Jung LEE; Jaesung RIM; Deun Sol JUNG; Minsu CHO; Sunghyun CHO
- Web page: https://patentgrep.com/patent/US20250157205A1

## CPC

- G06V 10/806: {of extracted features}
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G06V 10/771: Feature selection, e.g. selecting representative features from a multi-dimensional feature space
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method and apparatus with image enhancement using a base image are provided. The method includes: extracting, by a spatial feature extractor, intra-image spatial feature representations from respective image frames of a burst image set; generating inter-image temporal feature representations based on a local similarity between the spatial feature representations; determining temporal-spatial feature representations of the respective image frames by fusing the spatial feature representations with the temporal feature representations; selecting a base image frame from among the image frames based on the temporal-spatial feature representations; and generating an enhanced image by performing an image enhancement operation on the burst image set based on the base image frame.

## Claims

### Claim 1

An image enhancement method comprising:
extracting, by a spatial feature extractor, intra-image spatial feature representations from respective image frames of a burst image set;
generating inter-image temporal feature representations based on a local similarity between the spatial feature representations;
determining temporal-spatial feature representations of the respective image frames by fusing the spatial feature representations with the temporal feature representations;
selecting a base image frame from among the image frames based on the temporal-spatial feature representations; and
generating an enhanced image by performing an image enhancement operation on the burst image set based on the base image frame.

### Claim 2 (depends on 1)

The image enhancement method of claim 1, wherein the generating of the temporal feature representations comprises:
selecting a target spatial feature representation from among the spatial feature representations;
comparing window regions of the target spatial feature representation with spatially-corresponding search regions of the spatial feature representations; and
generating a temporal feature representation corresponding to the target spatial feature representation based on a result of the comparison, the temporal feature representation being included in the temporal feature representations.

### Claim 3 (depends on 2)

The image enhancement method of claim 2, wherein the comparing of the window regions of the target spatial feature representation with the spatially-corresponding search regions of the spatial feature representations comprises:
selecting a first spatial feature representation from among the spatial feature representations;
comparing a first window region of the window regions of the target spatial feature representation with a first search region spatially-corresponding to the first window region in the first spatial feature representation; and
comparing a second window region of the window regions of the target spatial feature representation with a second search region spatially-corresponding to the second window region in the first spatial feature representation.

### Claim 4 (depends on 1)

The image enhancement method of claim 1, wherein the temporal feature representations comprise:
motion information of the respective image frames.

### Claim 5 (depends on 1)

The image enhancement method of claim 1, wherein the determining of the temporal-spatial feature representations comprises:
causing a size of the spatial feature representations to equal a size of the temporal feature representations; and
determining the temporal-spatial feature representations based on an elementwise addition operation of feature values of the spatial feature representations and feature values of the temporal feature representations.

### Claim 6 (depends on 1)

The image enhancement method of claim 1, wherein
the generating of the temporal feature representations and the determining of the temporal-spatial feature representations are iteratively performed, and
the base image frame is selected based on final temporal-spatial feature representations obtained as a result of the iterations.

### Claim 7 (depends on 1)

The image enhancement method of claim 1, wherein the individual image frames have composite degradation.

### Claim 8 (depends on 1)

The image enhancement method of claim 1, wherein the individual image frames are captured with different respective exposure times.

### Claim 9 (depends on 1)

The image enhancement method of claim 1, wherein the selecting of the base image frame comprises:
generating a selection guide vector corresponding to the temporal-spatial feature representations, wherein elements of the selection guide vector respectively correspond to the frame images; and
selecting the base image frame from among the individual image frames based on values of the elements of the selection guide vector.

### Claim 10 (depends on 1)

The method of claim 1, wherein the temporal feature representations are feature maps generated by a first neural network, wherein the spatial-temporal features are feature maps generated by second neural network, and wherein a spatial-temporal feature map comprises spatial-temporal features of a corresponding image frame relative to other of the image frames.

### Claim 11

An electronic device comprising:
one or more processors; and
a memory storing instructions configured to cause the one or more processors to:
extract, by a spatial feature extractor, intra-image spatial feature representations from respective image frames of a burst image set;
generate inter-image temporal feature representations based on a local similarity between the spatial feature representations;
determine temporal-spatial feature representations of the respective image frames by fusing the spatial feature representations with the temporal feature representations;
select a base image frame from among the individual image frames based on the temporal-spatial feature representations; and
generate an enhanced image by performing an image enhancement operation on the burst image set based on the base image frame.

### Claim 12 (depends on 11)

The electronic device of claim 11, wherein, in order to generate the temporal feature representations, the instructions are further configured to cause the one or more processors to:
select a target spatial feature representation from among the spatial feature representations;
compare window regions of the target spatial feature representation with spatially-corresponding search regions of the spatial feature representations; and
generate a temporal feature representation corresponding to the target spatial feature representation based on a result of the comparison, the temporal feature representation being included in the temporal feature representations.

### Claim 13 (depends on 12)

The electronic device of claim 12, wherein, in order to compare the window regions of the target spatial feature representation with the search regions of the spatial feature representations, the instructions are further configured to cause the one or more processors to:
select a first spatial feature representation from among the spatial feature representations;
compare a first window region of the window regions of the target spatial feature representation with a first search region spatially-corresponding to the first window region in the first spatial feature representation; and
compare a second window region of the window regions of the target spatial feature representation with a second search region spatially-corresponding to the second window region in the first spatial feature representation.

### Claim 14 (depends on 11)

The electronic device of claim 11, wherein the temporal feature representations comprise:
motion information of the respective image frames.

### Claim 15 (depends on 11)

The electronic device of claim 11, wherein, in order to determine the temporal-spatial feature representations, the instructions may be further configured to cause the one or more processors to:
cause a size of the spatial feature representations to equal a size of the temporal feature representations; and
determine the temporal-spatial feature representations based on an elementwise addition operation of feature values of the spatial feature representations and feature values of the temporal feature representations.

### Claim 16 (depends on 11)

The electronic device of claim 11, wherein
the generating of the temporal feature representations and the determining of the temporal-spatial feature representations are iteratively performed, and
the base image frame is selected based on final temporal-spatial feature representations obtained as a result of the iterations.

### Claim 17 (depends on 11)

The electronic device of claim 11, wherein the individual image frames have composite degradation.

### Claim 18 (depends on 11)

The electronic device of claim 11, wherein the electronic device further comprises:
a camera configured to capture the burst image set.

### Claim 19 (depends on 18)

The electronic device of claim 18, wherein the camera is configured to:
capture the individual image frames of the burst image set with different respective exposure times.

### Claim 20 (depends on 11)

The electronic device of claim 11, wherein, in order to select the base image frame, the instructions are further configured to cause the one or more processors to:
generate a selection guide vector corresponding to the temporal-spatial feature representations, wherein elements of the selection guide vector respectively correspond to the frame images; and
select the base image frame from among the individual image frames based on values of the elements of the selection guide vector.
