# US 12,112,533 B2 · Method and apparatus for data calculation in neural network model, and image processing method and apparatus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,533 B2 - Document ID: US12112533B2 - Status: Granted - Kind: B2 - Application number: 17/499,497 - Filed: 2021-10-12 - Published: 2022-02-03 - Granted: 2024-10-08 - Earliest claimed: 2019-12-16 - Assignees: Tencent Technology (Shenzhen) Company Limited - Applicants: Tencent Technology (Shenzhen) Company Limited - Inventors: Hongrui Chen; Wenran Liu; Qifeng Chen; Haoyuan Li; Feng Li - Web page: https://patentgrep.com/patent/US12112533B2 ## CPC - G06V 10/955: {using specific electronic processors} - G06F 18/21: Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/063: using electronic means - G06N 3/08: Learning methods - G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining - G06V 10/82: using neural networks - G06V 40/107: {Static hand or arm} - G06V 40/161: {Detection; Localisation; Normalisation} - Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE ## Abstract A method and an apparatus for data calculation in a neural network model, and an image processing method and apparatus. The method for data calculation includes: reading weight data shared by a group of data processing of a data processing layer in a neural network model, into a GroupShared variable of a thread group of a graphics processing unit (GPU), dividing input data of the data processing layer based on the number of threads in the thread group, reading, for each group of input data, weight data corresponding to the group of input data for a data processing in the group of data processing from the GroupShared variable, and performing, by each thread in the thread group, the data processing by using a group of read input data and weight data corresponding to the group of input data, to obtain a calculation result corresponding to the group of input data. ## Claims ### Claim 1 A method for data calculation in a neural network model, performed by a computing device, the method comprising: reading weight data associated with a data processing layer in a neural network model into a GroupShared variable of a thread group of a graphics processing unit (GPU) dividing input data of the data processing layer into a plurality of groups of input data based on the number of threads in a depth dimension of the thread group; reading, for each group of input data, respective group weight data corresponding to each group of input data from the GroupShared variable; and performing, by each thread in the thread group, the data processing by using a group of read input data and respective weight data corresponding to the group of input data, to obtain a calculation result corresponding to the group of input data, wherein a data amount of the weight data corresponding to each group of input data after division for data processing is: Kw*Kh*C/Tz, wherein Tz is the number of threads in the depth dimension of the thread group, Kw is a width of a convolution kernel, Kh is a height of the convolution kernel, C is the number of input channels of the convolution layer, and C/Tz represents the number of input channels of convolution processed by each thread during a data processing. ### Claim 2 (depends on 1) The method according to claim 1, further comprising: storing calculation results corresponding to all groups of input data into the GroupShared variable. ### Claim 3 (depends on 1) The method according to claim 1, further comprising: performing preset processing on calculation results corresponding to the same output position among the calculation results corresponding to all the groups of input data, to obtain output data of the same output position. ### Claim 4 (depends on 3) The method according to claim 3, wherein the data processing layer comprises a convolution layer, and the performing preset processing on calculation results corresponding to the same output position among the calculation results corresponding to all the groups of input data, to obtain output data of the same output position comprises: adding up the calculation results corresponding to the same output position among the calculation results corresponding to all the groups of input data, and adding a convolution bias of the convolution layer, to obtain the output data of the same output position. ### Claim 5 (depends on 1) The method according to claim 1, further comprising: obtaining a target image; and processing the target image through the neural network model, to obtain a corresponding image processing result. ### Claim 6 (depends on 5) The method according to claim 5, wherein the neural network model comprises a target object recognition model, an image segmentation model, or an image classification model. ### Claim 7 An apparatus for data calculation in a neural network model, comprising at least one memory configured to store program code; and at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising: first weight data reading code configured to cause the at least one processor to read weight data associated with a data processing layer in a neural network model into a GroupShared variable of a thread group of a graphics processing unit (GPU) data division code configured to cause the at least one processor to divide input data of the data processing layer into a plurality of groups of input data based on the number of threads in a depth dimension of the thread group: second weight data reading code configured to cause the at least one processor to read, for each group of input data, respective group weight data corresponding to each group of input data from the GroupShared variable; and data calculation code configured to cause the at least one processor to enable each thread in the thread group to perform data processing by using a group of input data read by the second weight data reading code and respective weight data corresponding to the group of input data, to obtain a calculation result corresponding to the group of input data, wherein a data amount of the weight data corresponding to each group of input data after division for data processing is: Kw″Kh*C/Tz, wherein Tz is the number of threads in the depth dimension of the thread group, Kw is a width of a convolution kernel, Kh is a height of the convolution kernel, C is the number of input channels of the convolution layer, and C/Tz represents the number of input channels of convolution processed by each thread during a data processing. ### Claim 8 (depends on 7) The apparatus for data calculation according to claim 7, further comprising: storage code configured to cause the at least one processor to store calculation results corresponding to all the groups of input data into the GroupShared variable. ### Claim 9 (depends on 7) The apparatus for data calculation according to claim 7, further comprising: output data determining code configured to cause the at least one processor to perform preset processing on calculation results corresponding to the same output position among the calculation results corresponding to all the groups of input data, to obtain output data of the same output position. ### Claim 10 (depends on 9) The apparatus for data calculation according to claim 9, wherein the data processing layer comprises a convolution layer, and the output data determining code is further configured to cause the at least one processor to add up the calculation results corresponding to the same output position among the calculation results corresponding to all the groups of input data, and add a convolution bias of the convolution layer, to obtain the output data of the same output position. ### Claim 11 (depends on 7) The apparatus according to claim 7, further comprising image obtaining code configured to cause the at least one processor to obtain a target image; and image processing code configured to cause the at least one processor to process the target image through the neural network model, to obtain a corresponding image processing result. ### Claim 12 (depends on 11) The apparatus according to claim 11, wherein the neural network model comprises a target object recognition model, an image segmentation model, or an image classification model. ### Claim 13 A non-transitory computer-readable storage medium, storing a computer program that when executed by at least one processor causes the at least one processor to: read weight data associated with a data processing layer in a neural network model into a GroupShared variable of a thread group of a graphics processing unit (GPU); divide input data of the data processing layer into a plurality of groups of input data based on the number of threads in a depth dimension of the thread group; read, for each group of input data, respective group weight data corresponding to each group of input data from the GroupShared variable; and perform, by each thread in the thread group, the data processing by using a group of read input data and respective weight data corresponding to the group of input data, to obtain a calculation result corresponding to the group of input data, wherein a data amount of the weight data corresponding to each group of input data after division for data processing is: Kw*Kh*C/Tz, wherein Tz is the number of threads in the depth dimension of the thread group, Kw is a width of a convolution kernel, Kh is a height of the convolution kernel, C is the number of input channels of the convolution layer, and C/Tz represents the number of input channels of convolution processed by each thread during a data processing. ### Claim 14 (depends on 13) The non-transitory computer-readable storage medium according to claim 13, further configured to cause the at least one processor to: store calculation results corresponding to all groups of input data into the GroupShared variable. ### Claim 15 (depends on 13) The non-transitory computer-readable storage medium according to claim 13, further configured to cause the at least one processor to: perform preset processing on calculation results corresponding to the same output position among the calculation results corresponding to all the groups of input data, to obtain output data of the same output position. ### Claim 16 (depends on 15) The non-transitory computer-readable storage medium according to claim 15, wherein the data processing layer comprises a convolution layer, and the performing preset processing on calculation results corresponding to the same output position among the calculation results corresponding to all the groups of input data, to obtain output data of the same output position comprises: adding up the calculation results corresponding to the same output position among the calculation results corresponding to all the groups of input data, and adding a convolution bias of the convolution layer, to obtain the output data of the same output position. ### Claim 17 (depends on 13) The non-transitory computer-readable storage medium according to claim 13, further configured to cause the at least one processor to: obtain a target image; and process the target image through the neural network model, to obtain a corresponding image processing result.