# US 12,118,454 B2 · Neural network accelerator using logarithmic-based arithmetic > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,454 B2 - Document ID: US12118454B2 - Status: Granted - Kind: B2 - Application number: 18/537,570 - Filed: 2023-12-12 - Published: 2024-04-04 - Granted: 2024-10-15 - Earliest claimed: 2019-08-23 - Assignees: NVIDIA Corporation - Applicants: NVIDIA Corporation - Inventors: William James Dally; Rangharajan Venkatesan; Brucek Kurdo Khailany - Web page: https://patentgrep.com/patent/US12118454B2 ## CPC - G06N 3/063: using electronic means - G06F 7/4833: {Logarithmic number system} - G06F 17/16: Matrix or vector computation {, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization (matrix transposition G06F7/78)} - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/045: Combinations of networks - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Neural networks, in many cases, include convolution layers that are configured to perform many convolution operations that require multiplication and addition operations. Compared with performing multiplication on integer, fixed-point, or floating-point format values, performing multiplication on logarithmic format values is straightforward and energy efficient as the exponents are simply added. However, performing addition on logarithmic format values is more complex. Conventionally, addition is performed by converting the logarithmic format values to integers, computing the sum, and then converting the sum back into the logarithmic format. Instead, logarithmic format values may be added by decomposing the exponents into separate quotient and remainder components, sorting the quotient components based on the remainder components, summing the sorted quotient components to produce partial sums, and multiplying the partial sums by the remainder components to produce a sum. The sum may then be converted back into the logarithmic format. ## Claims ### Claim 1 A computer-implemented method, comprising: receiving a plurality of input values comprising image or sound data at a tensor core within a processor, wherein each input value in the plurality of input values is represented in a logarithmic format including a sign, a quotient component, and a remainder component, the tensor core is configured to implement at least one layer of a neural network model, and the at least one layer processes the plurality of input values by: generating a set of partial sums by sorting the quotient component for each input value according to the remainder component for the input value; summing the partial sums in the set of partial sums to produce a result that is a sum of the plurality of input values; using the result within at least one of an autonomous vehicle platform, financial modeling system, robotics system, speech recognition system, text recognition system, image recognition system, weather forecasting system, video analytics system, molecular simulation system, disease diagnosis system, data analytics system, molecular dynamics simulation system, factory automation system, real-time language translation system, online search optimization system, or personalized user recommendations system. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the quotient components are routed into separate accumulators to generate the set of partial sums. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein the generating comprises, for each quotient component, shifting a binary one by the quotient component to produce a quotient contribution input to one of the separate accumulators. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, wherein the quotient contributions input to each one of the separate accumulators are accumulated to produce the set of partial sums. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, further comprising, before summing the partial sums, scaling each partial sum in the set of partial sums by a constant corresponding to the accumulator that produced the partial sum. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, further comprising converting the sum of the plurality of input values into the logarithmic format including a sum quotient component and a sum remainder component. ### Claim 7 (depends on 6) The computer-implemented method of claim 6, wherein the converting comprises: determining the sum quotient component based on a position of a leading one within the sum; and quantizing a portion of the sum including the leading one and less significant bits of the sum to produce the sum remainder component. ### Claim 8 (depends on 7) The computer-implemented method of claim 7, wherein the quantizing comprises comparing the portion of the sum with a set of midpoints between pairs of possible remainder component values. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the set of partial sums is generated by sorting the quotient component for each input value according to at least one or more most-significant bits of the remainder component for the input value. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, wherein a subset of the remainder component includes at least one or more least-significant bits and the subset of the remainder component is divided by a base factor and summed with one to generate the partial sums. ### Claim 11 (depends on 10) The computer-implemented method of claim 10, wherein the subset of the remainder component is used to select one entry of a number of entries equal to each possible value of the subset of the remainder component to read a value equal to the subset of the remainder component divided by a base factor summed with one. ### Claim 12 (depends on 9) The computer-implemented method of claim 9, wherein a subset of the remainder component includes at least one or more least significant bits and the subset of the remainder component is divided by a base factor and multiplied by a scale factor to produce a product that is added to one to generate the partial sums. ### Claim 13 (depends on 9) The computer-implemented method of claim 9, wherein the quotient components are routed into separate accumulators to generate the set of partial sums. ### Claim 14 (depends on 1) The computer-implemented method of claim 1, wherein the plurality of input values are products computed as part of a convolution operation. ### Claim 15 A system, comprising: a memory storing a plurality of input values comprising image or sound data, wherein each input value in the plurality of input values is represented in a logarithmic format including a sign, a quotient component, and a remainder component; and a processor comprising: a tensor core configured to implement at least one layer of a neural network model, wherein the at least one layer processes the plurality of input values by: receiving the plurality of input values; generating a set of partial sums by sorting the quotient component for each input value according to the remainder component for the input value; and summing the partial sums in the set of partial sums to produce a result that is a sum of the plurality of input values; and using the result within at least one of an autonomous vehicle platform, financial modeling system, robotics system, speech recognition system, text recognition system, image recognition system, weather forecasting system, video analytics system, molecular simulation system, disease diagnosis system, data analytics system, molecular dynamics simulation system, factory automation system, real-time language translation system, online search optimization system, or personalized user recommendations system. ### Claim 16 (depends on 15) The system of claim 15, wherein the partial sums generation unit comprises a sorting unit configured to route the quotient components into separate accumulators to generate the set of partial sums. ### Claim 17 (depends on 16) The system of claim 16, wherein the processor is further configured to generate the set of partial sums by, for each quotient component, shift a binary one by the quotient component to produce a quotient contribution input to one of the separate accumulators. ### Claim 18 A processor comprising: a tensor core configured to implement at least one layer of a neural network model, wherein the at least one layer processes a plurality of input values comprising image or sound data by: receiving the plurality of input values, wherein each input value in the plurality of input values is represented in a logarithmic format including a sign, a quotient component, and a remainder component; generating a set of partial sums by sorting the quotient component for each input value according to the remainder component for the input value; summing the partial sums in the set of partial sums to produce a result that is a sum of the plurality of input values; and using the result within at least one of an autonomous vehicle platform, financial modeling system, robotics system, speech recognition system, text recognition system, image recognition system, weather forecasting system, video analytics system, molecular simulation system, disease diagnosis system, data analytics system, molecular dynamics simulation system, factory automation system, real-time language translation system, online search optimization system, or personalized user recommendations system. ### Claim 19 (depends on 18) The processor of claim 18, wherein the quotient components are routed into separate accumulators to generate the set of partial sums. ### Claim 20 (depends on 18) The processor of claim 18, further comprising converting the sum of the plurality of input values into the logarithmic format including a sum quotient component and a sum remainder component.