# US 2025/0159232 A1 · Image processing apparatus using neural network and image processing method using the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0159232 A1
- Document ID: US20250159232A1
- Status: Published application
- Kind: A1
- Application number: 18/654,604
- Filed: 2024-05-03
- Published: 2025-05-15
- Earliest claimed: 2023-11-15
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Paul OH; Seung Hoon Jee; Young Hun Sung; Ung Yeop Yang; Do Kwan Oh; Jun Hee Lee; Dong Won Jang; Jong Seong Choi; Chan Sol Hwang
- Web page: https://patentgrep.com/patent/US20250159232A1

## CPC

- H04N 19/513: Processing of motion vectors
- G06V 10/82: using neural networks
- H04N 19/117: Filters, e.g. for pre-processing or post-processing (sub-band filter banks H04N19/635)
- H04N 19/124: Quantisation
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/18: the unit being a set of transform coefficients
- H04N 19/91: Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An image encoding apparatus may include a memory storing one or more instructions; and a processor configured to execute the one or more instructions to: obtain an initial search position from a student neural network that has been trained through a knowledge distillation from a teacher neural network by inputting a current frame and a reference frame to the student neural network; and perform motion estimation based on the initial search position.

## Claims

### Claim 1

An image encoding apparatus comprising:
a memory storing one or more instructions; and
a processor configured to execute the one or more instructions to:
obtain an initial search position from a student neural network that has been trained through a knowledge distillation from a teacher neural network by inputting a current frame and a reference frame to the student neural network; and
perform motion estimation based on the initial search position.

### Claim 2 (depends on 1)

The image encoding apparatus of claim 1, wherein the student neural network is trained by inputting a first frame and a second frame into the teacher neural network to obtain a first initial search position, inputting the first frame and the second frame into the student neural network to obtain a second initial search position, and training the student neural network based on a loss function using a difference between the first initial search position and the second initial search position.

### Claim 3 (depends on 2)

The image encoding apparatus of claim 2, wherein the loss function comprises a multiplication of the difference between the first initial search position and the second initial search position by a loss weight.

### Claim 4 (depends on 3)

The image encoding apparatus of claim 3, wherein the loss weight is determined based on bitrate increasing rate information acquired by the teacher neural network.

### Claim 5 (depends on 4)

The image encoding apparatus of claim 4, wherein the bitrate increasing rate information is obtained by comparing an image compression result with the first initial search position applied and an image compression result without applying the first initial search position.

### Claim 6 (depends on 1)

The image encoding apparatus of claim 1, wherein the processor is further configured to output a predicted frame by performing motion compensation based on result of the motion estimation.

### Claim 7 (depends on 1)

The image encoding apparatus of claim 1, wherein the teacher neural network is located within the image encoding apparatus, or an external device located outside the image encoding apparatus.

### Claim 8 (depends on 1)

The image encoding apparatus of claim 1, wherein there are a plurality of teacher neural networks comprising the teacher neural network, and
the student neural network is trained by inputting a first frame and a second frame into each of the plurality of teacher neural networks, obtaining a plurality of initial search positions from the plurality of teacher neural networks, inputting the first frame and the second frame into the student neural network, obtaining the initial search position from the student neural network, and train the student neural network based on a loss function using differences between the plurality of initial search positions from the plurality of teacher neural networks and the initial search position from the student neural network.

### Claim 9 (depends on 2)

The image encoding apparatus of claim 2, wherein the loss function comprises a computation of a difference between the first initial search position and a ground truth label.

### Claim 10

An image encoding method comprising:
outputting an initial search position from a student neural network by inputting a current frame and a reference frame of an image into the student neural network that has been trained through a knowledge distillation from a teacher neural network; and
performing motion estimation based on the initial search position.

### Claim 11 (depends on 10)

The image encoding method of claim 10, wherein the outputting of the initial search position comprises:
obtaining a first initial search position from the teacher neural network by inputting a first frame and a second frame, into the teacher neural network;
obtaining a second initial search position from the student neural network by inputting the first frame and the second frame into the student neural network; and
training the student neural network based on a loss function using a difference between the first initial search position and the second initial search position.

### Claim 12 (depends on 11)

The image encoding method of claim 11, wherein the loss function comprises a multiplication of the difference between the first initial search position and the second initial search position by a loss weight.

### Claim 13 (depends on 12)

The image encoding method of claim 12, further comprising:
generating the loss weight based on bitrate increasing rate information acquired by the teacher neural network.

### Claim 14 (depends on 13)

The image encoding method of claim 13, further comprising:
obtaining the bitrate increasing rate information by comparing an image compression result of the student neural network with the first initial search position applied and an image compression result of the student neural network without applying the first initial search position.

### Claim 15 (depends on 10)

The image encoding method of claim 10, further comprising outputting a predicted frame by performing motion compensation based on result of the motion estimation.

### Claim 16 (depends on 11)

The image encoding method of claim 11, wherein the loss function comprises a computation of a difference between the first initial search position and a ground truth label.

### Claim 17

An image decoding apparatus comprising:
an entropy decoder configured to receive a bitstream and parse the received bitstream to obtain information necessary for image reconstruction;
an inverse quantizer configured to output a transform coefficient by inversely quantizing a quantized transform coefficient contained in the information;
an inverse transformer configured to obtain a residual block by inversely transforming the transform coefficient;
a predictor configured to generate a predicted block on the basis of information on prediction obtained from the entropy decoder;
an adder configured to generate a reconstructed block by adding the residual block and the predicted block; and
a filter configured to apply filtering to the reconstructed block,
wherein the bitstream is an encoded image signal based on a motion vector obtained using an initial search position output by inputting a current frame and a reference frame into a student neural network trained through a knowledge distillation technique using a teacher neural network.

### Claim 18

An image decoding method comprising:
receiving a bitstream and parsing the received bitstream to obtain information necessary for image reconstruction;
outputting a transform coefficient by inversely quantizing a quantized transform coefficient contained in the information;
obtaining a residual block by inversely transforming the transform coefficient;
generating a predicted block on the basis of information on prediction obtained from an entropy decoder;
generating a reconstructed block by adding the residual block and the predicted block; and
applying filtering to the reconstructed block,
wherein the bitstream is an encoded image signal based on a motion vector obtained using an initial search position output by inputting a current frame and a reference frame into a student neural network trained through a knowledge distillation technique using a teacher neural network.

### Claim 19

A non-transitory computer-readable storage medium storing a program that is executable by a processor to perform an image encoding method comprising:
outputting an initial search position from a student neural network by inputting a current frame and a reference frame of an image into the student neural network that has been trained through a knowledge distillation from a teacher neural network; and
performing motion estimation based on the initial search position.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, further comprising:
obtaining a bitrate increasing rate by comparing an image compression result of the student neural network using a teacher's initial search position obtained from the teacher neural network and an image compression result of the student neural network without using the teacher's initial search position; and
updating network parameters of the student neural network based on a loss function using the bitrate increasing rate as a loss weight.
