# US 2025/0225629 A1 · Method and electronic apparatus for correcting image distortion

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0225629 A1
- Document ID: US20250225629A1
- Status: Published application
- Kind: A1
- Application number: 18/902,106
- Filed: 2024-09-30
- Published: 2025-07-10
- Earliest claimed: 2024-01-09
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Younggeol KIM; Kwangpyo CHOI
- Web page: https://patentgrep.com/patent/US20250225629A1

## CPC

- G06T 5/80: Geometric correction
- G06T 7/11: Region-based segmentation
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/536: from perspective effects, e.g. by using vanishing points
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- H04N 23/67: Focus control based on electronic image sensor signals
- H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
- G06T 2207/20021: Dividing image into blocks, subimages or windows
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30201: Face
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method and electronic apparatus may be provided, the method includes obtaining a first image, generating a projection image based on the first image, in which perspective distortion of the first image is corrected, generating a cognitive map according to a plurality of objects in the first image, wherein the cognitive map divides the first image into a plurality of areas of a first plurality of types, generating first mesh information for correcting the first image based on the projection image and the cognitive map, wherein the first mesh information comprises movement information for a plurality of pixel positions, and generating an output image by correcting the first image based on the first mesh information.

## Claims

### Claim 1

A method, performed by an electronic apparatus, of correcting image distortion, comprising:
obtaining a first image;
generating a projection image based on the first image, in which perspective distortion of the first image is corrected;
generating a cognitive map according to a plurality of objects in the first image, wherein the cognitive map divides the first image into a plurality of areas of a first plurality of types;
generating first mesh information for correcting the first image based on the projection image and the cognitive map, wherein the first mesh information comprises movement information for a plurality of pixel positions; and
generating an output image by correcting the first image based on the first mesh information,
wherein the movement information is determined at different levels according to the first plurality of types.

### Claim 2 (depends on 1)

The method of claim 1, wherein the plurality of areas correspond to the plurality of objects, and
wherein the generating the cognitive map comprises:
identifying the plurality of objects in the first image;
extracting the plurality of areas;
determining the first plurality of types based on a second plurality of types of the plurality of objects; and
generating the cognitive map based on the first plurality of types.

### Claim 3 (depends on 2)

The method of claim 2, wherein the cognitive map comprises at least one of an area of interest with a high priority for correction of perspective distortion, a maintenance area in which no correction for perspective distortion is performed, or a low correction area with a low priority for correction of perspective distortion.

### Claim 4 (depends on 3)

The method of claim 3, wherein the determining the first plurality of types comprises:
determining a first area among the plurality of areas as the area of interest based on a first object in the first area comprising a face;
determining a second area among the plurality of areas as the maintenance area based on a second object in the second area comprising a geometric configuration, wherein the geometric configuration comprises at least one of a straight line, a curve, a polygon, a circle, or an ellipse; and
determining a third area among the plurality of areas as the low correction area based on a third object in the third area comprising a repeating pattern.

### Claim 5 (depends on 4)

The method of claim 4, wherein the first mesh information comprises first movement information and second movement information, and
wherein the generating the first mesh information comprises:
determining the first movement information such that a movement of a first pixel position in the area of interest is greater than a movement of a second pixel position in the low correction area; and
determining the second movement information such that a movement of a third pixel position in the low correction area is greater than a movement of a fourth pixel position in the maintenance area.

### Claim 6 (depends on 1)

The method of claim 1, wherein the generating the projection image comprises:
determining a vanishing point of the first image based on at least one of a field of view or a focal length of a lens through which the first image is captured; and
generating the projection image based on the vanishing point by correcting the first image.

### Claim 7 (depends on 1)

The method of claim 1, wherein the first mesh information comprises a movement vector, and
wherein the generating the first mesh information comprises:
dividing each of the first image and the projection image into a plurality of blocks; and
determining the movement vector based on a first block of the first image and a second block of the projection image corresponding to the first block.

### Claim 8 (depends on 1)

The method of claim 1, wherein the generating the output image comprises:
generating a second image by correcting the first image based on the first mesh information;
identifying whether the second image comprises a blank area; and
determining a pixel of the blank area.

### Claim 9 (depends on 1)

The method of claim 1, wherein the generating the projection image comprises:
determining depth information about an object in the first image; and
generating the projection image based on the depth information.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
obtaining a third image after the first image;
determining a motion vector between the third image and the first image; and
generating second mesh information for the third image based on correcting the first mesh information by using the motion vector.

### Claim 11

An electronic apparatus for correcting image distortion, comprising:
at least one processor including processing circuitry,
memory comprising one or more storage medium storing at least one instruction that, when executed by the at least one processor individually or collectively, cause the electronic apparatus to:
obtain a first image;
generate a projection image based on the first image in which perspective distortion of the first image is corrected;
generate a cognitive map according to a plurality of objects in the first image, wherein the cognitive map divides the first image into a plurality of areas of a first plurality of types;
generate first mesh information for correcting the first image based on the projection image and the cognitive map, wherein the first mesh information comprises movement information for a plurality of pixel positions; and
generate an output image by correcting the first image based on the first mesh information,
wherein the movement information is determined at different levels according to the first plurality of types.

### Claim 12 (depends on 11)

The electronic apparatus of claim 11, wherein the plurality of areas correspond to the plurality of objects, and
wherein the at least one instruction executed by the at least one processor individually or collectively, cause the electronic apparatus to:
identify the plurality of objects in the first image;
extract the plurality of areas;
determine the first plurality of types based on a second plurality of types of the plurality of objects; and
generate the cognitive map based on the first plurality of types.

### Claim 13 (depends on 12)

The electronic apparatus of claim 12, wherein the cognitive map comprises at least one of an area of interest with a high priority for correction of perspective distortion, a maintenance area in which no correction for perspective distortion is performed, or a low correction area with a low priority for correction of perspective distortion.

### Claim 14 (depends on 13)

The electronic apparatus of claim 13, wherein the at least one instruction executed by the at least one processor individually or collectively, cause the electronic apparatus to:
determine a first area among the plurality of areas as the area of interest based on a first object in the first area comprising a face;
determine a second area among the plurality of areas as the maintenance area based on a second object in the second area comprising a geometric configuration, wherein the geometric configuration comprises at least one of a straight line, a curve, a polygon, a circle, or an ellipse; and
determine a third area among the plurality of areas as the low correction area based on a third object in the third area comprising a repeating pattern.

### Claim 15 (depends on 14)

The electronic apparatus of claim 14, wherein the first mesh information comprises first movement information and second movement information, and
wherein the at least one instruction executed by the at least one processor individually or collectively, cause the electronic apparatus to:
determine the first movement information such that a movement of a first pixel position in the area of interest is greater than a movement of a second pixel position in the low correction area, and
determine the second movement information such that a movement of a third pixel position in the low correction area is greater than a movement of a fourth pixel position in the maintenance area.

### Claim 16 (depends on 11)

The electronic apparatus of claim 11, wherein the at least one instruction executed by the at least one processor individually or collectively, cause the electronic apparatus to:
determine a vanishing point of the first image based on at least one of a field of view or a focal length of a lens through which the first image is captured; and
generate the projection image based on the vanishing point by correcting the first image.

### Claim 17 (depends on 11)

The electronic apparatus of claim 11, wherein the first mesh information comprises a movement vector, and
wherein the at least one instruction executed by the at least one processor individually or collectively, cause the electronic apparatus to:
divide each of the first image and the projection image into a plurality of blocks; and
determine the movement vector based on a first block of the first image and a second block of the projection image corresponding to the first block.

### Claim 18 (depends on 11)

The electronic apparatus of claim 11, wherein the at least one instruction executed by the at least one processor individually or collectively, cause the electronic apparatus to:
generate a second image by correcting the first image based on the first mesh information;
identify whether the second image comprises a blank area; and
determine a pixel of the blank area.

### Claim 19 (depends on 11)

The electronic apparatus of claim 11, wherein the at least one instruction executed by the at least one processor individually or collectively, cause the electronic apparatus to:
determine depth information about an object in the first image; and
generate the projection image based on the depth information.

### Claim 20

A non-transitory computer-readable recording medium having instructions recorded thereon, that, when executed by at least one processor, cause the at least one processor to:
obtain a first image;
generate a projection image based on the first image in which perspective distortion of the first image is corrected;
generate a cognitive map according to a plurality of objects in the first image, wherein the cognitive map divides the first image into a plurality of areas of a first plurality of types;
generate first mesh information for correcting the first image based on the projection image and the cognitive map, wherein the first mesh information comprises movement information for a plurality of pixel positions; and
generate an output image by correcting the first image based on the first mesh information,
wherein the movement information is determined at different levels according to the first plurality of types.
