# US 2025/0168338 A1 · Deep learning based image partitioning for video compression

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0168338 A1
- Document ID: US20250168338A1
- Status: Published application
- Kind: A1
- Application number: 19/023,060
- Filed: 2025-01-15
- Published: 2025-05-22
- Earliest claimed: 2017-12-14
- Assignees: InterDigital VC Holdings, Inc.
- Applicants: InterDigital VC Holdings, Inc.
- Inventors: Franck Galpin; Fabrice LeLeannec; Fabien Racape; Sunil Jaiswal
- Web page: https://patentgrep.com/patent/US20250168338A1

## CPC

- H04N 19/119: Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 9/002: {using neural networks}
- H04N 19/105: Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
- H04N 19/176: the region being a block, e.g. a macroblock
- H04N 19/96: Tree coding, e.g. quad-tree coding
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A block of video data is split using one or more of several possible partition operations by using the partitioning choices obtained through use of a deep learning-based image partitioning. In at least one embodiment, the block is split in one or more splitting operations using a convolutional neural network. In another embodiment, inputs to the convolutional neural network come from pixels along the block's causal borders. In another embodiment, boundary information, such as the location of partitions in spatially neighboring blocks, is used by the convolutional neural network. Methods, apparatus, and signal embodiments are provided for encoding.

## Claims

### Claim at position 1

1-15. (canceled)

### Claim 16

A device for video processing, comprising:
a memory, and
a processor, configured to:
obtain an output of a first layer associated with a deep learning (DL) algorithm based on a block of image data and a causal pixel of the block;
concatenate the output of the first layer with compression information associated with the block;
input the concatenated output to a second layer associated with the DL algorithm;
partition the block into a plurality of smaller blocks based on an output of the second layer; and
predict the plurality of smaller blocks.

### Claim 17 (depends on 16)

The device of claim 16, wherein the DL algorithm comprises a convolutional neural network (CNN), and the compression information associated with the block comprises quantization information associated with the block.

### Claim 18 (depends on 17)

The device of claim 17, wherein the quantization information is associated with a quantization parameter of the block.

### Claim 19 (depends on 16)

The device of claim 16, wherein the block has a size of 64×64 pixels, and an input of the first layer associated with the DL algorithm has a size of 65×65 pixels.

### Claim 20 (depends on 16)

The device of claim 16, wherein the causal pixel is a pixel of a causal border of the block of image data, wherein the causal border is adjacent to the block and comprises a row of pixels on top of the block and a column of pixels to the left of the block.

### Claim 21 (depends on 20)

The device of claim 20, wherein the causal border of the block of image data is associated with a reconstructed image.

### Claim 22 (depends on 16)

The device of claim 16, wherein the output of the second layer comprises a vector of split possibilities.

### Claim 23 (depends on 22)

The device of claim 22, wherein the vector of split possibilities is a single component vector.

### Claim 24 (depends on 16)

The device of claim 16, wherein the processor is further configured to:
generate a residual based on the predicted plurality of smaller blocks; and
include the residual in a bitstream.

### Claim 25 (depends on 16)

The device of claim 16, wherein the processor is further configured to obtain an output of the second layer associated with the DL algorithm based on the concatenated output of the first layer associated with the DL algorithm.

### Claim 26

A method for video processing, comprising:
obtaining an output of a first layer associated with a deep learning (DL) algorithm based on a block of image data and a causal pixel of the block;
concatenating the output of the first layer with compression information associated with the block;
inputting the concatenated output to a second layer associated with the DL algorithm;
partitioning the block into a plurality of smaller blocks based on an output of the second layer; and
predicting the plurality of smaller blocks.

### Claim 27 (depends on 26)

The method of claim 26, wherein the DL algorithm comprises a convolutional neural network (CNN), and the compression information associated with the block comprises quantization information associated with the block.

### Claim 28 (depends on 27)

The method of claim 27, wherein the quantization information is associated with a quantization parameter of the block.

### Claim 29 (depends on 26)

The method of claim 26, wherein the block has a size of 64×64 pixels, and an input of the first layer associated with the DL algorithm has a size of 65×65 pixels.

### Claim 30 (depends on 26)

The method of claim 26, wherein the causal pixel is a pixel of a causal border of the block of image data, wherein the causal border is adjacent to the block and comprises a row of pixels on top of the block and a column of pixels to the left of the block.

### Claim 31 (depends on 30)

The method of claim 30, wherein the causal border of the block of image data is associated with a reconstructed image.

### Claim 32 (depends on 26)

The method of claim 26, wherein the output of the second layer comprises a vector of split possibilities.

### Claim 33 (depends on 26)

The method of claim 26, further comprising:
generating a residual based on the predicted plurality of smaller blocks; and
including the residual in a bitstream.

### Claim 34

A non-transitory computer readable medium containing program instructions that, when executed by a processor, cause the processor to perform a method comprising:
obtaining an output of a first layer associated with a deep learning (DL) algorithm based on a block of image data and a causal pixel of the block;
concatenating the output of the first layer with compression information associated with the block;
inputting the concatenated output to a second layer associated with the DL algorithm;
partitioning the block into a plurality of smaller blocks based on an output of the second layer; and
predicting the plurality of smaller blocks.

### Claim 35 (depends on 34)

The non-transitory computer readable medium of claim 34, wherein the DL algorithm comprises a convolutional neural network (CNN), and the compression information associated with the block comprises quantization information associated with the block.
