# US 2026/0019595 A1 · Method, apparatus, and medium for visual data processing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0019595 A1
- Document ID: US20260019595A1
- Status: Published application
- Kind: A1
- Application number: 19/335,653
- Filed: 2025-09-22
- Published: 2026-01-15
- Earliest claimed: 2023-03-22
- Assignees: Douyin Vision Co., Ltd.
- Applicants: Douyin Vision Co., Ltd.
- Inventors: Yaojun WU; Meng WANG
- Web page: https://patentgrep.com/patent/US20260019595A1

## CPC

- H04N 19/149: by estimating the code amount by means of a model, e.g. mathematical model or statistical model
- H04N 19/91: Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/048: Activation functions
- H04N 19/436: using parallelised computational arrangements
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Embodiments of the present disclosure provide a solution for visual data processing. A method for visual data processing is proposed. In the method, for a conversion between a current visual unit of visual data and a bitstream of the visual data, a probability representation of the current visual unit is determined based on a multistage context module. The conversion is performed based on the probability representation. The multistage context module at least comprises at least one prediction fusion network. The number of convolutional layers in the at least one prediction fusion network is less than or equal to a first threshold number.

## Claims

### Claim 1

A method for visual data processing, comprising:
determining, for a conversion between a current visual unit of visual data and a bitstream of the visual data, a probability representation of the current visual unit based on a multistage context module; and
performing the conversion based on the probability representation,
wherein the multistage context module at least comprises at least one prediction fusion network, and wherein the number of convolutional layers in the at least one prediction fusion network is less than or equal to a first threshold number.

### Claim 2 (depends on 1)

The method of claim 1, wherein the number of channels in at least one convolutional layer in the at least one prediction fusion network is less than or equal to a second threshold number, or
wherein the number of channels in at least one convolutional layer in the at least one prediction fusion network is larger than or equal to a second threshold number.

### Claim 3 (depends on 1)

The method of claim 1, wherein at least one activation layer in the at least one prediction fusion network is different from a first activation layer, and/or
wherein the at least one activation layer comprises a unified activation function, the unified activation function comprising one of: ReLU, LeakyReLU, GELU, or Sigmoid.

### Claim 4 (depends on 1)

The method of claim 1, wherein at least one parameter of the at least one convolutional layer in the at least one prediction fusion network is different from at least one parameter of a first convolutional layer.

### Claim 5 (depends on 1)

The method of claim 1, wherein the at least one prediction fusion network comprises a plurality of prediction fusion networks for a plurality of stages, and structures of the plurality of prediction fusion networks are same,
wherein for visual data representations in the plurality of stages, weights of the plurality of prediction fusion networks are different for respective stage operations, or wherein for visual data representations in the plurality of stages, weights of the plurality of prediction fusion networks are same for respective stage operations.

### Claim 6 (depends on 5)

The method of claim 5, wherein at least one layer in a prediction fusion network of the plurality of prediction fusion networks is removed, or
wherein the number of channels of at least one convolutional layer in the plurality of prediction fusion networks is a multiple of a positive integer, wherein the positive integer is 16 or 32, or
wherein the number of channels of at least one convolutional layer in the plurality of prediction fusion networks is 2n, n being a positive integer.

### Claim 7 (depends on 1)

The method of claim 1, wherein a plurality of prediction fusion network structures is supported, a first syntax element in the bitstream indicating a target prediction fusion network structure of the plurality of prediction fusion network structures to be used for the at least one prediction fusion network,
wherein a second syntax element in the bitstream indicates the number of the plurality of prediction fusion network structures.

### Claim 8 (depends on 1)

The method of claim 1, wherein a syntax element in the bitstream indicates whether to use the at least one prediction fusion network or a further prediction fusion network with a structure different from the at least one prediction fusion network, and/or
wherein a flag in the bitstream indicates that the at least one prediction fusion network is applied to at least one of: luma or chroma.

### Claim 9 (depends on 1)

The method of claim 1, wherein the multistage context module further comprises at least one multistage context network,
wherein the number of channels in the at least one multistage context network is less than or equal to a third threshold number,
wherein an input channel and an output channel of the multistage context network are updated, or wherein the number of channels is updated based on a change of input information.

### Claim 10 (depends on 9)

The method of claim 9, wherein a kernel size of the at least one multistage context network is greater than or equal to a threshold size, wherein the kernel size is 4×4,
wherein the at least one multistage context network applies a first multistage mask convolutional pattern different from a second multistage mask convolutional pattern, and/or
wherein the multistage context network comprises four stages or nine stages.

### Claim 11 (depends on 1)

The method of claim 1, wherein the multistage context module further comprises a conditional context network, and wherein the number of convolutional layers in the conditional context network is less than or equal to a fourth threshold number.

### Claim 12 (depends on 11)

The method of claim 11, wherein the number of channels in at least one convolutional layer in the conditional context network is less than or equal to a threshold number, or
wherein the number of channels in at least one convolutional layer in the conditional context network is larger than or equal to a threshold number, or
wherein the number of channels in at least one convolutional layer in the conditional context network is one of: a multiple of M, M being 16 or 32, or 2n, n being a positive integer.

### Claim 13 (depends on 11)

The method of claim 11, wherein at least one activation layer in the conditional context network is different from a first activation layer, wherein the at least one activation layer comprises a unified activation function, the unified activation function comprising one of: ReLU, LeakyReLU, GELU, or Sigmoid.

### Claim 14 (depends on 11)

The method of claim 11, wherein at least one parameter of at least one convolutional layer in the conditional context network is different from at least one parameter of a first convolutional layer, or
wherein for a plurality of groups of latents grouped by a channel dimension, the conditional context network with a same structure and different weights is used to process latents of respective groups, or
wherein for a plurality of groups of latents grouped by a channel dimension, the conditional context network with a same structure and same weights is used to process latents of respective groups.

### Claim 15 (depends on 11)

The method of claim 11, wherein a kernel size of a convolutional layer in the conditional context network comprises 4×4, and/or
wherein a plurality of conditional context network structures is supported, a syntax element in the bitstream indicating a target conditional context network structure to be used for the conditional context network, wherein a further syntax element in the bitstream indicated the number of the plurality of conditional context network structures, and/or
wherein a syntax element in the bitstream indicates whether to use the conditional context network or a further conditional context network with a structure different from the conditional context network, and/or
wherein a flag in the bitstream indicates that the conditional context network is applied to at least one of: luma or chroma.

### Claim 16 (depends on 1)

The method of claim 1, wherein a stride of convolution of at least one convolutional layer in the multistage context module is an integer greater than or equal to 2.

### Claim 17 (depends on 1)

The method of claim 1, wherein the conversion comprises decoding the current visual unit from the bitstream, or
wherein the conversion comprises encoding the current visual unit into the bitstream.

### Claim 18

An apparatus for visual data processing comprising a processor and a non-transitory memory with instructions thereon, wherein the instructions upon execution by the processor, cause the processor to:
determine, for a conversion between a current visual unit of visual data and a bitstream of the visual data, a probability representation of the current visual unit based on a multistage context module; and
perform the conversion based on the probability representation,
wherein the multistage context module at least comprises at least one prediction fusion network, and wherein the number of convolutional layers in the at least one prediction fusion network is less than or equal to a first threshold number.

### Claim 19

A non-transitory computer-readable storage medium storing instructions that cause a processor to perform acts comprising:
determining, for a conversion between a current visual unit of visual data and a bitstream of the visual data, a probability representation of the current visual unit based on a multistage context module; and
performing the conversion based on the probability representation,
wherein the multistage context module at least comprises at least one prediction fusion network, and wherein the number of convolutional layers in the at least one prediction fusion network is less than or equal to a first threshold number.

### Claim 20

A non-transitory computer-readable recording medium storing a bitstream of visual data which is generated by a method performed by an apparatus for visual data processing, wherein the method comprises:
determining a probability representation of a current visual unit of the visual data based on a multistage context module; and
generating the bitstream based on the probability representation,
wherein the multistage context module at least comprises at least one prediction fusion network, and wherein the number of convolutional layers in the at least one prediction fusion network is less than or equal to a first threshold number.
