# US 12,182,901 B2 · Interleaving and rearrangement to reduce a number of dimensions of image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,901 B2 - Document ID: US12182901B2 - Status: Granted - Kind: B2 - Application number: 17/991,416 - Filed: 2022-11-21 - Published: 2023-03-16 - Granted: 2024-12-31 - Earliest claimed: 2021-04-26 - Assignees: Tencent Technology (Shenzhen) Company Limited - Applicants: Tencent Technology (Shenzhen) Company Limited - Inventors: Tong Ai; Feng Li; Haoyuan Li - Web page: https://patentgrep.com/patent/US12182901B2 ## CPC - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06T 1/60: Memory management - G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An image data processing method is provided. In the method, a first image data set that includes image data arranged according to a first data format is acquired. The first data format has a first number of dimensions. Interleaving and rearrangement are performed on the image data in the first image data set based on a convolution operation to obtain a second image data set. The image data in the second image data set is arranged according to a second data format. The second data format has a second number of dimensions. The second number of dimensions of the second data format is less than first number of dimensions of the first data format. A target output result is generated based on the convolution operation performed on the second image data set and a second weight data set. Apparatus and non-transitory computer-readable storage medium counterpart embodiments are also contemplated. ## Claims ### Claim 1 An image data processing method, comprising: acquiring a first image data set that includes image data arranged according to a first data format, the first data format having a first number of dimensions; performing interleaving and rearrangement on a channel dimension of the image data in the first image data set based on a convolution operation to obtain a second image data set with a reduction in a quantity of channels in the channel dimension of the first number of dimensions, the image data in the second image data set being arranged according to a second data format, the second data format having a second number of dimensions, the second number of dimensions of the second data format being less than the first number of dimensions of the first data format; and generating a target output result based on the convolution operation performed on the second image data set and a second weight data set. ### Claim 2 (depends on 1) The method according to claim 1, wherein the dimensions of the second data format are a subset of the dimensions of the first data format. ### Claim 3 (depends on 1) The method according to claim 1, wherein the first data format is N1×C1×H1×W1, N1 representing a quantity of image data subsets included in the first image data set, C1 representing the quantity of channels in each of the image data subsets in the first image data set, H1 representing a data height in each of the image data subsets in the first image data set, and W1 representing a data width in each of the image data subsets in the first image data set. ### Claim 4 (depends on 3) The method according to claim 3, wherein the second data format is N1×H2×W2, H2 representing a data height in each of the image data subsets in the second image data set, and W2 representing a data width in each of the image data subsets in the second image data set. ### Claim 5 (depends on 4) The method according to claim 4, wherein the performing the interleaving and the rearrangement comprises: dividing the image data of each of M1 channels in the first image data set into Si groups of the image data, M1≤C1; and performing the interleaving and the rearrangement on the image data of the M1 channels in each of the Si groups of the image data to obtain the second image data set. ### Claim 6 (depends on 5) The method according to claim 5, wherein each of the Si groups includes one of the image data from each of the M1 channels. ### Claim 7 (depends on 5) The method according to claim 5, wherein the generating the target output result includes performing the convolution operation on the second image data set and the second weight data set to obtain a fourth image data set, the target output result including the fourth image data set, the performing the interleaving and the rearrangement on the image data in the first image data set includes performing the interleaving and the rearrangement on the image data of the M1 channels in each of the S1 groups of the image data in which the image data of each of the M1 channels in the first image data set is divided into a group, and M1≤C1. ### Claim 8 (depends on 7) The method according to claim 7, wherein the generating the target output result operation comprises: acquiring C2 groups of the image data in the second image data set, each group of the image data including a plurality of pieces of the image data located in a same channel in the first image data set, and each group of the image data is obtained by offsetting one address from a storage address of a previous group of the image data in the C2 groups of the image data; and performing the convolution operation on the C2 groups of the image data and N2×C2 groups of weight data in the second weight data set to obtain N2 groups of the image data in the fourth image data set, each group of the weight data having a same data structure as each group of the image data. ### Claim 9 (depends on 8) The method according to claim 8, wherein the second weight data set is obtained by performing interleaving and rearrangement on weight data in a first weight data set, the weight data in the first weight data set being arranged according to the data format of N2×C2×H3×W3, N2 representing the quantity of weight data subsets included in the first weight data set, C2 representing the quantity of channels in each of the weight data subsets, H3 representing the data height in each of the weight data subsets, and W3 representing the data width in each of the weight data subsets. ### Claim 10 (depends on 4) The method according to claim 4, further comprising: acquiring a first weight data set, the first weight data set including weight data that is arranged according to a data format of N2×C2×H3×W3, N2 representing a quantity of weight data subsets included in the first weight data set, C2 representing a quantity of channels in each of the weight data subsets, H3 representing a data height in each of the weight data subsets, and W3 representing a data width in each of the weight data subsets; and performing interleaving and rearrangement on the weight data in the first weight data set to obtain the second weight data set, the weight data in the second weight data set being arranged according to a data format of H4×W4, H4 representing a data height of the weight data in the second weight data set, and W4 representing a data width of the weight data in the second weight data set. ### Claim 11 (depends on 10) The method according to claim 10, wherein the performing the interleaving and the rearrangement on the weight data in the first weight data set comprises: dividing the weight data of each of M2 weight data subsets in the first weight data set into S2 groups of the weight data, M2≤N2; and performing interleaving and rearrangement on the weight data from the M2 weight data in each of the S2 groups of the weight data to obtain the second weight data set. ### Claim 12 (depends on 10) The method according to claim 10, wherein a value of N2 is a quantity of output channels of a convolution kernel, a value of C2 is a quantity of input channels of the convolution kernel, the convolution operation is performed by using the convolution kernel, and each of the weight data subsets includes weight data in the C2 input channels. ### Claim 13 (depends on 1) The method according to claim 1, further comprising: storing the first image data set and the second image data set in a first memory space; and storing the second weight data set in a second memory space, wherein the first memory space is independent of the second memory space. ### Claim 14 An information processing apparatus, comprising: processing circuitry configured to: acquire a first image data set that includes image data arranged according to a first data format, the first data format having a first number of dimensions; perform interleaving and rearrangement on a channel dimension of the image data in the first image data set based on a convolution operation to obtain a second image data set with a reduction in a quantity of channels in the channel dimension of the first number of dimensions, the image data in the second image data set being arranged according to a second data format, the second data format having a second number of dimensions, the second number of dimensions of the second data format being less than first number of dimensions of the first data format; and generate a target output result based on the convolution operation performed on the second image data set and a second weight data set. ### Claim 15 (depends on 14) The information processing apparatus according to claim 14, wherein the dimensions of the second data format are a subset of the dimensions of the first data format. ### Claim 16 (depends on 14) The information processing apparatus according to claim 14, wherein the first data format is N1×C1×H1×W1, N1 representing a quantity of image data subsets included in the first image data set, C1 representing the quantity of channels in each of the image data subsets in the first image data set, H1 representing a data height in each of the image data subsets in the first image data set, and W1 representing a data width in each of the image data subsets in the first image data set. ### Claim 17 (depends on 16) The information processing apparatus according to claim 16, wherein the second data format is N1×H2×W2, H2 representing a data height in each of the image data subsets in the second image data set, and W2 representing a data width in each of the image data subsets in the second image data set. ### Claim 18 (depends on 17) The information processing apparatus according to claim 17, wherein the processing circuitry is configured to: divide the image data of each of M1 channels in the first image data set into Si groups of the image data, M1≤C1; and perform the interleaving and the rearrangement on the image data of the M1 channels in each of the S1 groups of the image data to obtain the second image data set. ### Claim 19 (depends on 18) The information processing apparatus according to claim 18, wherein each of the S1 groups includes one of the image data from each of the M1 channels. ### Claim 20 A non-transitory computer-readable storage medium, storing instructions which when executed by a processor cause the processor to perform: acquiring a first image data set that includes image data arranged according to a first data format, the first data format having a first number of dimensions; performing interleaving and rearrangement on a channel dimension of the image data in the first image data set based on a convolution operation to obtain a second image data set with a reduction in a quantity of channels in the channel dimension of the first number of dimensions, the image data in the second image data set being arranged according to a second data format, the second data format having a second number of dimensions, the second number of dimensions of the second data format being less than first number of dimensions of the first data format; and generating a target output result based on the convolution operation performed on the second image data set and a second weight data set.