# US 12,182,687 B2 · Data representation for dynamic precision in neural network cores > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,687 B2 - Document ID: US12182687B2 - Status: Granted - Kind: B2 - Application number: 16/157,852 - Filed: 2018-10-11 - Published: 2020-04-16 - Granted: 2024-12-31 - Earliest claimed: 2018-10-11 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: John V. Arthur; Andrew S. Cassidy; Myron D. Flickner; Pallab Datta; Hartmut Penner; Rathinakumar Appuswamy; Jun Sawada; Dharmendra S. Modha; Steven K. Esser; Brian Taba; Jennifer Klamo - Web page: https://patentgrep.com/patent/US12182687B2 ## CPC - G06N 3/063: using electronic means - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/048: Activation functions - G06N 3/0495: Quantised networks; Sparse networks; Compressed networks - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Systems for neural network computation are provided. A neural network processor comprises a plurality of neural cores. The neural network processor has one or more processor precisions per activation. The processor is configured to accept data having a processor feature dimension. A transformation circuit is coupled to the neural network processor, and is adapted to: receive an input data tensor having an input precision per channel at one or more features; transform the input data tensor from the input precision to the processor precision; divide the input data into a plurality of blocks, each block conforming to one of the processor feature dimensions; provide each of the plurality of blocks to one of the plurality of neural cores. The neural network processor is adapted to compute, by the plurality of neural cores, output of one or more neural network layers. ## Claims ### Claim 1 A method comprising: receiving an input data tensor at a neural network processor comprising a plurality of neural cores, the input data tensor having feature dimensions at an input bit precision, the neural network processor being configured for one or more feature dimensions at one or more processor bit precisions; determining a number of blocks by dividing the input bit precision of the input data tensor by one of the one or more processor bit precisions; determining one of the one or more processor bit precisions; based on the determination of the one of the one or more processor bit precisions, transforming the input data tensor from the input bit precision to the determined one of the one or more processor bit precisions, wherein transforming the input data tensor comprises removing one or more least significant bits of the input data tensor, wherein a number of the one or more least significant bits is based on the one or more processor bit precisions; dividing the input data tensor into the number of blocks, each block conforming to one of the feature dimensions of the neural network processor; providing each of the number of blocks to one of the plurality of neural cores; and computing, by the plurality of neural cores, output of one or more neural network layers. ### Claim 2 (depends on 1) The method of claim 1, wherein the input data tensor comprises an image. ### Claim 3 (depends on 1) The method of claim 1, the neural network processor being configured for a predetermined number of features, wherein transforming the input data tensor comprises dividing input features into a plurality of feature sets, each having less than or equal to the predetermined number of features. ### Claim 4 (depends on 1) The method of claim 1, wherein dividing the input data tensor comprises zero-padding the number of blocks in one of the feature dimensions of the input data tensor to conform with one of the feature dimensions of the neural network processor. ### Claim 5 (depends on 1) The method of claim 1, wherein dividing the input data tensor comprises packing the input data tensor. ### Claim 6 (depends on 5) The method of claim 5, wherein packing the input data tensor comprises: reorganizing input features to load unused feature dimensions of the processor with data from non-feature dimensions of the input features. ### Claim 7 (depends on 1) The method of claim 1, wherein the neural network processor is configured to: compute a plurality of fixed precision partial sums; and combine the plurality of fixed precision partial sums into complete sums. ### Claim 8 (depends on 7) The method of claim 7, wherein the plurality of fixed precision partial sums are intermediate results. ### Claim 9 (depends on 8) The method of claim 8, wherein the intermediate results are weighted sums of a subset of inputs. ### Claim 10 (depends on 7) The method of claim 7, wherein the neural network processor is configured to iteratively compute a partial sum from the plurality of fixed precision partial sums. ### Claim 11 A system comprising: a neural network processor comprising a plurality of neural cores, the neural network processor having one or more processor precisions per activation, the neural network processor being configured to accept data having a feature dimension; a transformation circuit coupled to the neural network processor, and adapted to: receive an input data tensor having an input precision per channel at one or more features; determine one of the one or more processor precisions; determining a number of blocks by dividing the input bit precision of the input data tensor by one of the one or more processor bit precisions; based on the determination of the one of the one or more processor precisions, transform the input data tensor from the input precision to the determined one of the one or more processor precisions, wherein transforming the input data tensor comprises removing one or more least significant bits of the input data tensor, wherein a number of the one or more least significant bits is based on the one or more processor bit precisions; divide the input data tensor into the number of blocks, each block conforming to one of the feature dimensions of the neural network processor; and provide each of the number of blocks to one of the plurality of neural cores; and wherein the neural network processor is adapted to compute, by the plurality of neural cores, output of one or more neural network layers. ### Claim 12 (depends on 11) The system of claim 11, wherein the input data tensor comprises an image. ### Claim 13 (depends on 11) The system of claim 11, wherein transforming the input data tensor comprises dividing each channel into a plurality of values having precisions less than or equal to the one of the processor precisions. ### Claim 14 (depends on 11) The system of claim 11, wherein dividing the input data tensor comprises zero-padding the number of blocks in a feature dimension of the input data tensor to conform with one of the feature dimensions of the neural network processor. ### Claim 15 (depends on 11) The system of claim 11, wherein dividing the input data tensor comprises packing the input data tensor. ### Claim 16 (depends on 15) The wherein of claim 15, wherein packing the input data tensor comprises: reorganizing input features to load unused feature dimensions of the processor with data from non-feature dimensions of the input features. ### Claim 17 (depends on 11) The system of claim 11, wherein the neural network processor is configured to: compute a plurality of fixed precision partial sums; and combine the plurality of fixed precision partial sums into complete sums. ### Claim 18 (depends on 17) The system of claim 17, wherein the plurality of fixed precision partial sums are intermediate results. ### Claim 19 (depends on 18) The system of claim 18, wherein the intermediate results are weighted sums of a subset of inputs. ### Claim 20 (depends on 17) The system of claim 17, wherein the neural network processor is configured to iteratively compute a partial sum from the plurality of fixed precision partial sums. ### Claim 21 A system comprising: a neural network processor comprising a plurality of neural cores, the neural network processor being configured for one or more feature dimensions at one or more processor bit precisions; and a transformation circuit coupled to the neural network processor, and adapted to: receive an input data tensor having feature dimensions at an input bit precision; determine one of the one or more processor bit precisions; determining a number of blocks by dividing the input bit precision of the input data tensor by one of the one or more processor bit precisions; based on the determination of the one of the one or more processor bit precisions, transform the input data tensor from the input bit precision to the determined one of the one or more processor bit precisions, wherein transforming the input data tensor comprises and removing one or more least significant bits of the input data tensor, and wherein a number of the one or more least significant bits is based on the one or more processor bit precisions; divide the input data tensor into the number of blocks, each block conforming to one of the feature dimensions of the neural network processor; and provide each of the number of blocks to one of the plurality of neural cores; and wherein the plurality of neural cores is adapted to compute output of one or more neural network layers. ### Claim 22 A method comprising: receiving an input data tensor, the input data tensor having an input precision per channel at one or more features; determining a processor precision of a neural network processor; determining a number of blocks by dividing the input bit precision of the input data tensor by one of the one or more processor bit precisions; based on the determination of the processor precision, transforming the input data tensor from the input precision to the processor precision of the neural network processor, wherein transforming the input data tensor comprises removing one or more least significant bits of the input data tensor, and wherein a number of the one or more least significant bits is based on the processor precision, the neural network processor having a plurality of neural cores; the neural network processor being configured to accept data having a feature dimension; dividing the input data tensor into the number of blocks, each block conforming to one of the feature dimensions of the neural network processor; and providing each of the number of blocks to one of the plurality of neural cores, and wherein the neural network processor is adapted to compute, by the plurality of neural cores, output of one or more neural network layers.