# US 2022/0044104 A1 · Method and apparatus for forward computation of neural network, and computer-readable storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0044104 A1
- Document ID: US20220044104A1
- Status: Published application
- Kind: A1
- Application number: 17/507,127
- Filed: 2021-10-21
- Published: 2022-02-10
- Earliest claimed: 2019-12-16
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Wen Ran LIU; Hong Rui CHEN; Hao Yuan LI; Qi Feng CHEN; Feng LI
- Web page: https://patentgrep.com/patent/US20220044104A1

## CPC

- G06N 3/063: using electronic means
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method and an apparatus for forward computation of a neural network are provided. The method includes: obtaining input data and weight data of at least one data processing layer; storing the input data by using a first texture storage structure to obtain first texture data; storing the weight data by using a second texture storage structure to obtain second texture data; and performing data processing of the at least one data processing layer based on the first texture data and the second texture data, to obtain output data of the at least one data processing layer. For the at least one data processing layer in the neural network, the input data and the weight data are respectively stored in corresponding texture storage structures.

## Claims

### Claim 1

A method for forward computation of a neural network, performed by a computer device, for data processing at least one data processing layer in the neural network, the method comprising:
obtaining input data and weight data of the at least one data processing layer;
storing the input data by using a first texture storage structure to obtain first texture data, a plurality of data elements in the input data corresponding to a first index of the first texture data, and the computer device accessing data in the first texture data by using the first index as a unit;
storing the weight data by using a second texture storage structure to obtain second texture data, a plurality of data elements in the weight data corresponding to a second index of the second texture data, and the computer device accessing data in the second texture data by using the second index as a unit; and
performing data processing of the at least one data processing layer based on the first texture data and the second texture data, to obtain output data of the at least one data processing layer.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the first texture data comprises at least one texel indexed in at least two dimensions, each of the at least one texel has a plurality of channels, and each of the plurality of channels is used for storing a data element in the input data, and
the storing the input data by using the first texture storage structure to obtain the first texture data comprises sequentially storing data elements in the input data into the plurality of channels of the at least one texel.

### Claim 3 (depends on 2)

The method according to claim 2, further comprising:
obtaining a pixel width, a pixel height, and a channel quantity of the input data; and
determining magnitudes in the at least two dimensions and a channel quantity of the first texture storage structure based on the pixel width, the pixel height, and the channel quantity of the input data and a dimension quantity of the first index of the first texture storage structure, a product of the determined magnitudes in the at least two dimensions and the determined channel quantity being not less than a product of the pixel width, the pixel height, and the channel quantity of the input data.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the first texture storage structure is a 3D texture storage structure, and the at least two dimensions comprise a height, a width, and a depth, and
the determining the magnitudes in the at least two dimensions and the channel quantity of the first texture storage structure comprises:
using the pixel width of the input data as the width of the first texture storage structure, and using the pixel height of the input data as the height of the first texture storage structure; and
determining the depth of the first texture storage structure based on the channel quantity of the input data and the channel quantity of each texel in the first texture storage structure.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the determining the depth of the first texture storage structure based on the channel quantity of the input data and the channel quantity of is based on the following expression:
d
1
=
⌊
(
c
1
+
s
1
-
1
)
s
1
⌋
wherein d1 represents the depth of the first texture storage structure, c1 represents the channel quantity of the input data, s1 represents the channel quantity of each texel in the first texture storage structure, and └ ┘ represents a round-down operation symbol.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the storing the weight data by using the second texture storage structure to obtain the second texture data comprises:
obtaining parameter data related to a data amount of the weight data;
determining, based on the parameter data related to the data amount of the weight data and a dimension quantity of the second index of the second texture storage structure, magnitudes in at least two dimensions and a channel quantity of the second texture storage structure that enable the second texture storage structure to store a data amount not less than the data amount of the weight data; and
sequentially storing data elements in the weight data into channels of texels in the second texture storage structure, to obtain the second texture data.

### Claim 7 (depends on 6)

The method according to claim 6, wherein the second texture storage structure is a 3D texture storage structure, and the at least two dimensions of the second texture storage structure comprise a height, a width, and a depth of the second texture storage structure, and
based on the parameter data related to the data amount of the weight data comprising a pixel width and a pixel height of the weight data, and an input channel quantity and an output channel quantity of the at least one data processing layer, the determining the magnitudes in the at least two dimensions and the channel quantity of the second texture storage structure comprises:
determining the height and the width of the second texture storage structure based on one a first one of the input channel quantity and the output channel quantity of the at least one data processing layer, and the pixel width and the pixel height of the weight data; and
determining the depth of the second texture storage structure based on a second one of the input channel quantity and the output channel quantity of the at least one data processing layer, and the channel quantity of each texel in the second texture storage structure.

### Claim 8 (depends on 7)

The method according to claim 7, wherein the determining the height and the width of the second texture storage structure comprises:
using the pixel width of the weight data as the width of the second texture storage structure; and
using a product of the pixel height of the weight data and the channel quantity as the height of the first texture storage structure, and
the determining the depth of the second texture storage structure comprises:
determining the depth of the second texture storage structure based on the following expression:
d
2
=
⌊
(
c
2
+
s
2
-
1
)
s
2
⌋
wherein d2 represents the depth of the second texture storage structure, c2 represents the other channel quantity, s2 represents the channel quantity of each texel in the second texture storage structure, and └ ┘ represents a round-down operation symbol.

### Claim 9 (depends on 6)

The method according to claim 6, wherein the second texture storage structure is a 3D texture storage structure, and the at least two dimensions of the second texture storage structure comprise a height, a width, and a depth of the second texture storage structure; and
based on the parameter data related to the data amount of the weight data comprising a pixel width and a pixel height of the weight data, and an input channel quantity or an output channel quantity of the at least one data processing layer, the determining the magnitudes in the at least two dimensions and the channel quantity of the second texture storage structure comprises:
using the pixel width of the weight data as the width of the second texture storage structure, using the pixel height of the weight data as the height of the second texture storage structure, and determining the depth of the second texture storage structure based on the input channel quantity or the output channel quantity of the at least one data processing layer, and the channel quantity of each texel in the second texture storage structure.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the determining the depth of the second texture storage structure based on the input channel quantity or the output channel quantity of the at least one data processing layer, and the channel quantity of each texel in the second texture storage structure comprises:
determining the depth of the second texture storage structure based on the following expression:
d
3
=
⌊
(
c
3
+
s
3
-
1
)
s
3
⌋
wherein d3 represents the depth of the second texture storage structure, c3 represents the input channel quantity or the output channel quantity, s3 represents the channel quantity of each texel in the second texture storage structure, and └ ┘ represents a round-down operation symbol.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the performing data processing of the at least one data processing layer is performed by using at least one computation unit in a graphics processing unit (GPU), and
the performing data processing of the at least one data processing layer based on the first texture data and the second texture data, to obtain output data of the at least one data processing layer comprises:
reading, by using computation units, input data stored at a first index location in the first texture data, and reading, by using the computation units, weight data that is in the second texture data and stored at a second index location corresponding to the first index location; and
performing, by using the computation units, data processing on the input data and corresponding weight data that are read, to obtain the output data of the at least one data processing layer.

### Claim 12 (depends on 1)

The method according to claim 1, wherein the first texture storage structure or the second texture storage structure is any one of the following:
a red, green, blue, alpha (RGBA)-channel 3D texture storage structure;
an RGB-channel 3D texture storage structure;
an RGBA-channel 2D texture storage structure; and
an RGB-channel 2D texture storage structure.

### Claim 13

An apparatus configured to perform data processing of at least one data processing layer in a neural network, the apparatus comprising:
at least one memory configured to store program code; and
at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising:
data obtaining code configured to cause at least one of the at least one processor to obtain input data and weight data of the at least one data processing layer;
data storage code configured to cause at least one of the at least one processor to store the input data by using a first texture storage structure to obtain first texture data, and store the weight data by using a second texture storage structure to obtain second texture data, a plurality of data elements in the input data corresponding to a first index of the first texture data, a plurality of data elements in the weight data corresponding to a second index of the second texture data, the apparatus accessing data in the first texture data by using the first index as a unit and the second texture data by using the second index as a unit; and
data processing code configured to cause at least one of the at least one processor perform data processing of the at least one data processing layer based on the first texture data and the second texture data, to obtain output data of the at least one data processing layer.

### Claim 14 (depends on 13)

The apparatus according to claim 13, wherein the first texture data comprises at least one texel indexed in at least two dimensions, each of the at least one texel has a plurality of channels, and each of the plurality of channels is configured to store a data element in the input data, and
the data storage code is further configured to cause at least one of the at least one processor to sequentially store data elements in the input data into the plurality of channels of the at least one texel.

### Claim 15 (depends on 13)

The apparatus according to claim 13, wherein the data storage code is further configured to cause at least one of the at least one processor to:
obtain a pixel width, a pixel height, and a channel quantity of the input data; and
determine magnitudes in at least two dimensions and a channel quantity of the first texture storage structure based on the pixel width, the pixel height, and the channel quantity of the input data and a dimension quantity of the first index of the first texture storage structure, a product of the determined magnitudes in the at least two dimensions and the determined channel quantity being not less than a product of the pixel width, the pixel height, and the channel quantity of the input data.

### Claim 16 (depends on 13)

The apparatus according to claim 13, wherein the data storage code is further configured to cause at least one of the at least one processor to:
obtain parameter data related to a data amount of the weight data;
determine, based on the parameter data related to the data amount of the weight data and a dimension quantity of the second index of the second texture storage structure, magnitudes in at least two dimensions and a channel quantity of the second texture storage structure that enable the second texture storage structure to store a data amount not less than the data amount of the weight data; and
sequentially store data elements in the weight data into channels of texels in the second texture storage structure, to obtain the second texture data.

### Claim 17 (depends on 16)

The apparatus according to claim 16, wherein the second texture storage structure is a 3D texture storage structure, and the at least two dimensions of the second texture storage structure comprise a height, a width, and a depth of the second texture storage structure, and
the data storage code is further configured to cause at least one of the at least one processor to determine the magnitudes in the at least two dimensions and the channel quantity of the second texture storage structure, based on the parameter data related to the data amount of the weight data comprising a pixel width and a pixel height of the weight data, and an input channel quantity and an output channel quantity of the at least one data processing layer, by:
determining the height and the width of the second texture storage structure based on one a first one of the input channel quantity and the output channel quantity of the at least one data processing layer, and the pixel width and the pixel height of the weight data; and
determining the depth of the second texture storage structure based on a second one of the input channel quantity and the output channel quantity of the at least one data processing layer, and the channel quantity of each texel in the second texture storage structure.

### Claim 18 (depends on 17)

The apparatus according to claim 17, wherein
the data storage code is further configured to cause at least one of the at least one processor to determine the height and the width of the second texture storage structure by:
using the pixel width of the weight data as the width of the second texture storage structure; and
using a product of the pixel height of the weight data and the channel quantity as the height of the first texture storage structure, and
the determining the depth of the second texture storage structure comprises:
determining the depth of the second texture storage structure based on the following expression:
d
2
=
⌊
(
c
2
+
s
2
-
1
)
s
2
⌋
wherein d2 represents the depth of the second texture storage structure, c2 represents the other channel quantity, s2 represents the channel quantity of each texel in the second texture storage structure, and └ ┘ represents a round-down operation symbol.

### Claim 19 (depends on 13)

The apparatus for forward computation according to claim 13, wherein the data processing code is further configured to cause at least one of the at least one processor to:
read, by using at least one computation unit in a graphics processing unit (GPU), input data stored at a first index location in the first texture data, and read, by using the computation units, weight data that is in the second texture data and stored at a second index location corresponding to the first index location; and
perform, by using the computation units, data processing on the input data and corresponding weight data that are read, to obtain the output data of the at least one data processing layer.

### Claim 20

A non-transitory computer-readable storage medium, storing a computer program, the computer program being executable by at least one processor to cause the at least one processor to perform a method for forward computation of a neural network for data processing at least one data processing layer in the neural network, the method comprising:
obtaining input data and weight data of the at least one data processing layer;
storing the input data by using a first texture storage structure to obtain first texture data, a plurality of data elements in the input data corresponding to a first index of the first texture data, and the computer device accessing data in the first texture data by using the first index as a unit;
storing the weight data by using a second texture storage structure to obtain second texture data, a plurality of data elements in the weight data corresponding to a second index of the second texture data, and the computer device accessing data in the second texture data by using the second index as a unit; and
performing data processing of the at least one data processing layer based on the first texture data and the second texture data, to obtain output data of the at least one data processing layer.