# US 12,015,428 B2 · MAC processing pipeline using filter weights having enhanced dynamic range, and methods of operating same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,015,428 B2 - Document ID: US12015428B2 - Status: Granted - Kind: B2 - Application number: 17/074,670 - Filed: 2020-10-20 - Published: 2021-05-06 - Granted: 2024-06-18 - Earliest claimed: 2019-11-05 - Assignees: Flex Logix Technologies, Inc. - Applicants: Flex Logix Technologies, Inc. - Inventors: Frederick A Ware; Cheng C. Wang - Web page: https://patentgrep.com/patent/US12015428B2 ## CPC - H03M 7/24: Conversion to or from floating-point codes - G06F 7/5443: {Sum of products (for applications thereof, see the relevant places, e.g. G06F17/10, H03H17/00)} - G06F 2207/3808: concerning the type of numbers or the way they are handled - G06F 2207/3824: Accepting both fixed-point and floating-point numbers - H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system and/or an integrated circuit including: (a) a multiplier-accumulator execution pipeline including multiplier-accumulator circuits to process image data, using associated filter weights, via a plurality of multiply and accumulate operations and (b) first data format conversion circuitry including (i) inputs to receive filter weights of a plurality of sets of filter weights, wherein each set includes a plurality of filter weights each having a block-scaled fraction data format, (ii) conversion circuitry, coupled to the inputs, to convert the filter weights of each set from the block-scaled fraction data format to a floating point data format, and (iii) outputs to output the filter weights having the floating point data format. In operation, the multiplier-accumulator circuits of the pipeline are configured to perform the plurality of multiply and accumulate operations using (a) the image data and (b) the filter weights having the floating point data format. ## Claims ### Claim 1 An integrated circuit comprising: a first memory to store a plurality of sets of filter weights; first data format conversion circuitry having: inputs to receive the plurality of sets of filter weights from the first memory, wherein each of the sets of filter weights includes a plurality of filter weights having a block-scaled fraction data format; floating point format conversion circuitry, coupled to the inputs of the first data format conversion circuitry, to convert the filter weights of each set of filter weights from the block-scaled fraction data format to a floating point data format; and outputs to output the filter weights of each set of filter weights having the floating point data format; and a second memory coupled to the outputs of the first data format conversion circuitry to receive and store the filter weights of each set of filter weights having the floating point data format; and a multiplier-accumulator execution pipeline having a plurality of multiplier-accumulator circuits, each of the multiplier-accumulator circuits having: first inputs to receive image data; second inputs coupled to the second memory to receive a respective one of the sets of filter weights having the floating point data format; and multiply-accumulate circuitry to generate output data by performing multiply and accumulate operations using the image data received via the first inputs and the respective one of the sets of filter weights received via the second inputs. ### Claim 2 (depends on 1) The integrated circuit of claim 1 wherein the second memory comprises static random access memory (SRAM). ### Claim 3 (depends on 1) The integrated circuit of claim 1 further comprising second data format conversion circuitry to: receive first filter data having constituent floating-point-format filter weights; convert the floating-point-format filter weights of the first filter data to filter weights having the block-scaled fraction data format; and output the filter weights having the block-scaled fraction data format to the first memory for storage therein as the plurality of sets of filter weights. ### Claim 4 (depends on 3) The integrated circuit of claim 3 wherein the first filter data having constituent floating-point-format filter weights comprises a plurality of sets of floating-point-format filter weights, and wherein the second data format conversion circuitry further comprises circuitry to perform the following operations with respect to each one of the sets of floating-point-format filter weights: determine a largest exponent of the constituent filter weights of the one of the sets of the floating-point-format filter weights, right shift a fraction field of each constituent filter weight of the one of the sets of the floating-point-format filter weights having a smaller exponent than the largest exponent, round the fraction field of each constituent filter weight of the one of the sets of the floating-point-format filter weights to a block-scaled fraction (BSF) precision, and two's complement the fraction field of each constituent filter weight of the one of the sets of the floating-point-format filter weights having a negative fraction field. ### Claim 5 (depends on 3) The integrated circuit of claim 3 wherein: each filter weight received via the second inputs of the multiplier-accumulation circuits and having the floating point data format is constituted by a first number of bits; and each of the constituent floating-point-format filter weights of the first filter data received within the second data conversion circuitry is constituted by a second number of bits less than the first number of bits. ### Claim 6 (depends on 3) The integrated circuit of claim 3 wherein the second data conversion circuitry comprises circuitry to determine a largest exponent of the constituent filter weights of each one of the sets of the floating-point-format filter weights. ### Claim 7 (depends on 6) The integrated circuit of claim 6 wherein the second data format conversion circuitry further comprises circuitry to right shift a fraction field of each constituent filter weight of the one of the sets of the floating-point-format filter weights having a smaller exponent than the largest exponent, round the fraction field to a block-scaled fraction (BSF) precision, and, if negative, perform a two's complement operation on the fraction field. ### Claim 8 (depends on 1) The integrated circuit of claim 1 wherein both the first and second memories comprise static random access memory (SRAM). ### Claim 9 (depends on 1) The integrated circuit of claim 1 further comprising second data format conversion circuitry to: receive first filter data having constituent integer-format filter weights; convert the integer-format filter weights of the first filter data to a block-scaled fraction data format; and output the filter weights having the block-scaled fraction data format to the first memory for storage therein as the plurality of sets of filter weights. ### Claim 10 An integrated circuit comprising: a first memory to store a plurality of sets of filter weights; first data format conversion circuitry having: inputs to receive the plurality of sets of filter weights from the first memory, wherein each of the sets of filter weights includes a plurality of filter weights having a block-scaled fraction data format; Winograd conversion circuitry, coupled to the inputs of the first data format conversion circuitry, to convert each of the sets of filter weights to a corresponding Winograd set of filter weights; floating point format conversion circuitry, coupled to the Winograd conversion circuitry, to convert the filter weights of each Winograd set of filter weights to a floating point data format; and outputs to output the filter weights of each Winograd set of filter weights having the floating point data format; a second memory coupled to the outputs of the first data format conversion circuitry to receive and store the filter weights of each Winograd set of filter weights having the floating point data format; and a multiplier-accumulator execution pipeline having a plurality of multiplier-accumulator circuits, each of the multiplier-accumulator circuits having: first inputs to receive image data; second inputs coupled to the second memory to receive a respective one of the Winograd sets of filter weights having the floating point data format; and multiply-accumulate circuitry to generate output data by performing multiply and accumulate operations using the image data received via the first inputs and the respective one of the Winograd sets of filter weights received via the second inputs. ### Claim 11 (depends on 10) The integrated circuit of claim 10 wherein the second memory comprises static random access memory (SRAM). ### Claim 12 (depends on 10) The integrated circuit of claim 10 further comprising second data format conversion circuitry to: receive first filter data having constituent floating-point-format filter weights; convert the floating-point-format filter weights of the first filter data to filter weights having the block-scaled fraction data format; and output the filter weights having the block-scaled fraction data format to the first memory for storage therein as the plurality of sets of filter weights. ### Claim 13 (depends on 12) The integrated circuit of claim 12 wherein: the second data format conversion circuitry comprises circuitry to determine a largest exponent of the constituent filter weights of each one of the sets of the floating-point-format filter weights. ### Claim 14 (depends on 13) The integrated circuit of claim 13 wherein the second data format conversion circuitry further comprises circuitry to right shift a fraction field of each constituent filter weight of the one of the sets of the floating-point-format filter weights having a smaller exponent than the largest exponent, round the fraction field to a block-scaled fraction (BSF) precision, and, if negative, perform a two's complement operation on the fraction field. ### Claim 15 (depends on 10) The integrated circuit of claim 10 further comprising second data format conversion circuitry to: receive first filter data having constituent integer-format filter weights; convert the integer-format filter weights of the first filter data to a block-scaled fraction data format, and output the filter weights having the block-scaled fraction data format to the first memory for storage therein as the plurality of sets of filter weights.