# US 12,645,755 B2 · Data processing in a machine learning computer > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,645,755 B2 - Document ID: US12645755B2 - Status: Granted - Kind: B2 - Application number: 18/066,627 - Filed: 2022-12-15 - Published: 2023-06-15 - Granted: 2026-06-02 - Earliest claimed: 2021-12-15 - Assignees: Graphcore Limited - Applicants: Graphcore Limited - Inventors: Godfrey Da Costa; Badreddine Noune; Daniel Justus; Carlo Luschi - Web page: https://patentgrep.com/patent/US12645755B2 ## CPC - G06F 18/211: Selection of the most significant subset of features - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06F 18/22: Matching criteria, e.g. proximity measures - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/048: Activation functions - G06N 3/084: Backpropagation, e.g. using gradient descent - G06V 10/82: using neural networks - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A computer-implemented method comprising: processing data in a neural network to compute a network tensor comprising a plurality of tensor elements represented in an initial numerical format; computing a histogram of tensor elements; selecting a target numerical format, the target numerical format having a lower precision than the initial numerical format; evaluating a metric based on the histogram of tensor elements and the target numerical format, the metric indicating a degree of accuracy of a representation of the network tensor in the target numerical format; and based on the evaluated metric, converting the plurality of tensor elements from the initial numerical format to the target numerical format. ## Claims ### Claim 1 A computer-implemented method comprising: processing data in a neural network to compute a network tensor comprising a plurality of tensor elements represented in an initial numerical format; computing a histogram of tensor elements; selecting a target numerical format, the target numerical format having a lower precision than the initial numerical format; evaluating a metric based on the histogram of tensor elements and the target numerical format, the metric indicating a degree of accuracy of a representation of the network tensor in the target numerical format; and based on the evaluated metric, converting the plurality of tensor elements from the initial numerical format to the target numerical format. ### Claim 2 (depends on 1) The method of claim 1, wherein the target numerical format is associated with a fixed range of representable exponent values. ### Claim 3 (depends on 1) The method of claim 1, wherein the target numerical format is selected by selecting a value of an exponent bias of the target numerical format. ### Claim 4 (depends on 1) The method of claim 1, wherein the target numerical format is selected by selecting a value of a number of exponent bits of the target numerical format. ### Claim 5 (depends on 1) The method of claim 1, wherein the network tensor comprises at least one of: weights; activations; gradients of a loss function with respect to weights; and gradients of a loss function with respect to activations. ### Claim 6 (depends on 1) The method of claim 1, comprising: selecting multiple target numerical formats, and for each target numerical format, evaluating the metric based on the histogram and the target numerical format to generate a respective score for each target numerical format; and comparing the scores for each target numerical format to identify an optimal format parameter, converting the plurality of tensor elements from the initial numerical format to the target numerical format. ### Claim 7 (depends on 2) The method of claim 2, wherein the format parameter comprises an exponent bias and a number of exponent bits, the method comprising: selecting a first combination of an exponent bias and a number of exponent bits and evaluating the metric based on the histogram of tensor elements and the numerical format corresponding to the selected exponent bias and number of exponent bits, thereby generating a first score for the first combination; selecting a second combination of an exponent bias and a number of exponent bits, wherein at least one of the exponent bias and the number of exponent bits differs between the first combination and the second combination, and evaluating the metric based on the histogram of tensor elements and the numerical format corresponding to the second combination of exponent bias and number of exponent bits, thereby generating a second score for that combination of exponent bias and number of exponent bits; comparing the first and second score to determine a best combination of the first and second combinations; and storing a representation of the network tensor in the numerical format corresponding to the best combination. ### Claim 8 (depends on 1) The method of claim 1, wherein the metric is one of: format density; mean-square error; KL divergence; and signal-to-noise ratio. ### Claim 9 (depends on 1) The method of claim 1, comprising applying an operation to the plurality of network tensor elements in the target numerical format. ### Claim 10 (depends on 9) The method of claim 9, wherein the operation comprises at least one of a matrix multiplication and a convolution operation. ### Claim 11 (depends on 1) The method of claim 1, wherein the target numerical format is an eight-bit format. ### Claim 12 (depends on 11) The method of claim 11, further comprising: computing a second network tensor comprising a plurality of second tensor elements represented in a second initial numerical format; computing a second histogram of second tensor elements; selecting a second target numerical format, the second target numerical format having a lower precision than the second initial numerical format; evaluating a metric based on the second histogram and the second target numerical format, the metric indicating a degree of accuracy of a representation of the network tensor in the second target numerical format; and based on the evaluated metric, converting the plurality of tensor elements of the second network tensor from the second initial numerical format to the second target numerical format. ### Claim 13 (depends on 11) The method of claim 11, wherein the target numerical format and the second target numerical format are associated with different respective ranges of representable exponent values. ### Claim 14 A computer system comprising memory and one or more processors configured to: process data in a neural network to compute a network tensor comprising a plurality of tensor elements represented in an initial numerical format; compute a histogram of tensor elements; select a target numerical format, the target numerical format having a lower precision than the initial numerical format; evaluate a metric based on the histogram of tensor elements and the target numerical format, the metric indicating a degree of accuracy of a representation of the network tensor in the target numerical format; and based on the evaluated metric, convert the plurality of tensor elements from the initial numerical format to the target numerical format. ### Claim 15 (depends on 14) The computer system of claim 14, wherein the one or more processors comprises a plurality of processors, wherein each processor is configured to process a respective subset of the neural network. ### Claim 16 (depends on 14) The computer system of claim 14, wherein the one or more processors comprises a plurality of processors, wherein each processor is configured to process a respective subset of the data. ### Claim 17 A non-transitory computer-readable storage medium storing computer program instructions which when executed on one or more processors perform the steps of: processing data in a neural network to compute a network tensor comprising a plurality of tensor elements represented in an initial numerical format; computing a histogram of tensor elements; selecting a target numerical format, the target numerical format having a lower precision than the initial numerical format; evaluating a metric based on the histogram of tensor elements and the target numerical format, the metric indicating a degree of accuracy of a representation of the network tensor in the target numerical format; and based on the evaluated metric, converting the plurality of tensor elements from the initial numerical format to the target numerical format.