# US 2023/0069072 A1 · Image processing apparatus and operation method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0069072 A1
- Document ID: US20230069072A1
- Status: Published application
- Kind: A1
- Application number: 17/888,170
- Filed: 2022-08-15
- Published: 2023-03-02
- Earliest claimed: 2021-09-02
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Iljun AHN; Soomin KANG; Hanul SHIN; Tammy LEE
- Web page: https://patentgrep.com/patent/US20230069072A1

## CPC

- G06T 5/70: Denoising; Smoothing
- G06T 7/73: using feature-based methods
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T 5/60: using machine learning, e.g. neural networks
- G06T 7/00: Image analysis
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/82: using neural networks
- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An image processing apparatus may include: a memory storing one or more instructions; and a processor configured to execute the one or more instructions stored in the memory, wherein the processor executes the one or more instructions to use one or more convolutional neural networks to: determine first parameters, based on position information of first pixels included in a first region of a first image and position information of samples of the first image; obtain sample values of the samples by applying the first parameters to pixel values of the first pixels; obtain feature information corresponding to the first region by performing a convolution operation between the sample values and a kernel of the one or more convolutional neural networks; and generate a second image, based on the feature information.

## Claims

### Claim 1

An image processing apparatus comprising:
at least one memory storing one or more instructions; and
at least one processor configured to execute the one or more instructions stored in the at least one memory,
wherein the at least one processor is configured to execute the one or more instructions to use one or more convolutional neural networks to:
determine first parameters based on position information of first pixels included in a first region of a first image and position information of samples of the first image;
obtain sample values of the samples by applying the first parameters to pixel values of the first pixels;
obtain feature information corresponding to the first region by performing a convolution operation between the sample values and a kernel of the one or more convolutional neural networks; and
generate a second image, based on the feature information.

### Claim 2 (depends on 1)

The image processing apparatus of claim 1, wherein
the first pixels comprise: a center pixel located at a center of the first region; and neighboring pixels of the center pixel,
the samples comprise the center pixel, and the samples other than the center pixel are each equidistant from the center pixel.

### Claim 3 (depends on 1)

The image processing apparatus of claim 1, wherein a number of the samples is equal to a number of weights in the kernel.

### Claim 4 (depends on 1)

The image processing apparatus of claim 1, wherein the at least one processor is further configured to execute the one or more instructions to determine the first parameters, based on at least one of information about distances between the first pixels and the samples or information about areas of regions formed based on the first pixels and the samples.

### Claim 5 (depends on 1)

The image processing apparatus of claim 1, wherein the at least one processor is further configured to execute the one or more instructions to obtain a first sample value of a first sample by applying the first parameters to pixel values of second pixels located in a neighborhood of the first sample among the first pixels.

### Claim 6 (depends on 5)

The image processing apparatus of claim 5, wherein the at least one processor is further configured to execute the one or more instructions to determine the first parameters, based on distances between the second pixels and the first sample.

### Claim 7 (depends on 5)

The image processing apparatus of claim 5, wherein the at least one processor is further configured to execute the one or more instructions to determine the first parameters, based on information about areas of regions formed by the second pixels and the first sample.

### Claim 8 (depends on 5)

The image processing apparatus of claim 5, wherein a sum of the first parameters applied to the second pixels is 1.

### Claim 9

An image processing apparatus comprising:
at least one memory storing one or more instructions; and
at least one processor configured to execute the one or more instructions stored in the at least one memory,
wherein the at least one processor is configured to execute the one or more instructions to use one or more convolutional neural networks to:
determine first parameters, based on position information of first pixels included in a first region of a first image and position information of samples;
obtain adjusted weights by applying the first parameters to weights in a kernel of the one or more convolutional neural networks;
obtain feature information corresponding to the first region by performing a convolution operation between pixel values of the first pixels and the adjusted weights; and
generate a second image, based on the feature information.

### Claim 10 (depends on 9)

The image processing apparatus of claim 9, wherein
the first pixels comprise: a center pixel located at a center of the first region; and neighboring pixels of the center pixel,
the samples comprise the center pixel, and the samples other than the center pixel are each equidistant from the center pixel.

### Claim 11 (depends on 9)

The image processing apparatus of claim 9, wherein the at least one processor is further configured to execute the one or more instructions to determine the first parameters, based on at least one of information about distances between the first pixels and the samples or information about areas of regions formed based on the first pixels and the samples.

### Claim 12 (depends on 11)

The image processing apparatus of claim 11, wherein the at least one processor is further configured to execute the one or more instructions to determine the first parameters, based on the weights in the kernel.

### Claim 13

An operation method of an image processing apparatus for processing an image by using one or more convolutional neural networks, the operation method comprising:
determining first parameters, based on position information of first pixels included in a first region of a first image and position information of samples of the first image;
obtaining sample values of the samples by applying the first parameters to pixel values of the first pixels;
obtaining feature information corresponding to the first region by performing a convolution operation between the sample values and a kernel of the one or more convolutional neural networks; and
generating a second image, based on the feature information.

### Claim 14 (depends on 13)

The operation method of claim 13, wherein
the first pixels comprise: a center pixel located at a center of the first region; and neighboring pixels of the center pixel,
the samples comprise the center pixel, and the samples other than the center pixel are each equidistant from the center pixel.

### Claim 15 (depends on 13)

The operation method of claim 13, wherein a number of the samples is equal to a number of weights in the kernel.

### Claim 16 (depends on 13)

The operation method of claim 13, wherein the determining of the first parameters comprises determining the first parameters, based on at least one of information about distances between the first pixels and the samples or information about areas of regions formed based on the first pixels and the samples.

### Claim 17 (depends on 13)

The operation method of claim 13, wherein the obtaining of the sample values comprises obtaining a first sample value of a first sample by applying the first parameters to pixel values of second pixels located in a neighborhood of the first sample among the first pixels.

### Claim 18 (depends on 17)

The operation method of claim 17, wherein the determining of the first parameters comprises determining the first parameters, based on distances between the second pixels and the first sample.

### Claim 19 (depends on 17)

The operation method of claim 17, wherein the determining of the first parameters comprises determining the first parameters, based on information about areas of regions formed by the second pixels and the first sample.

### Claim 20 (depends on 17)

The operation method of claim 17, wherein a sum of the first parameters applied to the second pixels is 1.
