# US 12,118,460 B2 · Discovery of hardware characteristics of deep learning accelerators for optimization via compiler > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,460 B2 - Document ID: US12118460B2 - Status: Granted - Kind: B2 - Application number: 17/092,033 - Filed: 2020-11-06 - Published: 2022-05-12 - Granted: 2024-10-15 - Earliest claimed: 2020-11-06 - Assignees: Micron Technology, Inc. - Applicants: Micron Technology, Inc. - Inventors: Aliasger Tayeb Zaidy; Marko Vitez; Eugenio Culurciello; Jaime Cummins; Andre Xian Ming Chang - Web page: https://patentgrep.com/patent/US12118460B2 ## CPC - G06F 8/443: {Optimisation} - G06N 3/08: Learning methods - G06F 8/41: Compilation - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/063: using electronic means - G06N 3/105: {Shells for specifying net layout} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Systems, devices, and methods related to a Deep Learning Accelerator and memory are described. For example, an integrated circuit device may be configured to execute instructions with matrix operands and configured with random access memory. A computing device running a compiler can interact and/or probe an integrated circuit device to identify hardware characteristics of the integrated circuit device in performing matrix computations. The compiler can generate and optimize a result of compilation from a description of an artificial neural network based at least in part on the hardware characteristics of the integrated circuit device. The result of compilation can include first data representative of parameters of the artificial neural network and second data representative of instructions executable by the integrated circuit device to generate an output of the artificial neural network based on the first data and an input to the artificial neural network. ## Claims ### Claim 1 A method, comprising: transmitting, by a computing device, one or more commands to an integrated circuit device to cause the integrated circuit device to generate a response by exercising matrix computations identified by each respective command in the one or more commands, wherein the response for the respective command is indicative of hardware characteristics of the integrated circuit device in performing the matrix computations during the exercising; determining, by the computing device, hardware characteristics of the integrated circuit device from the response for the respective command; generating, by the computing device, a result of compilation from data representative of a description of an artificial neural network based at least in part on the hardware characteristics of the integrated circuit device determined from the response to the one or more commands, the result of compilation including first data representative of parameters of the artificial neural network and second data representative of first instructions executable by the integrated circuit device to generate an output of the artificial neural network based on the first data and an input to the artificial neural network; and mapping, based on a specification indicating a capability of at least one different type of integrated circuit device, the result of the compilation generated using the description of the artificial neural network into at least one different compilation result for the at least one different type of integrated circuit device to generate second instructions executable on the at least one different type of integrated circuit device. ### Claim 2 (depends on 1) The method of claim 1, wherein the hardware characteristics of the integrated circuit device identify features, options, behaviors, performances, or latency of at least one processing unit of the integrated circuit device, or any combination thereof. ### Claim 3 (depends on 2) The method of claim 2, wherein the transmitting of the one or more commands to the integrated circuit device includes loading test programs into the integrated circuit device to receive responses of executing the test programs in the integrated circuit device. ### Claim 4 (depends on 3) The method of claim 3, further comprising: determining the hardware characteristics from the responses of executing the test programs in the integrated circuit device. ### Claim 5 (depends on 3) The method of claim 3, further comprising: determining the hardware characteristics from timestamps associated with executions of the test programs in the integrated circuit device. ### Claim 6 (depends on 3) The method of claim 3, further comprising: identifying a specification of the integrated circuit device, among a plurality of predetermined specifications of integrated circuit devices, based on the responses of executing the test programs in the integrated circuit device. ### Claim 7 (depends on 3) The method of claim 3, further comprising: probing hardware options of a matrix processing unit of the integrated circuit based on the responses of executing the test programs in the integrated circuit device. ### Claim 8 (depends on 3) The method of claim 3, further comprising: generating the test programs based on specifications of a plurality of hardware platforms of integrated circuit devices configured to perform matrix computations. ### Claim 9 (depends on 3) The method of claim 3, further comprising: transforming the result of compilation from the description of the artificial neural network to improve, based on the hardware characteristics of the integrated circuit device, performance of the result when executed in the integrated circuit device. ### Claim 10 A computing device, comprising: memory; and at least one microprocessor coupled to the memory and configured to: probe an integrated circuit device to identify hardware characteristics of the integrated circuit device in performing matrix computations, generate a result of compilation from data representative of a description of an artificial neural network based at least in part on the hardware characteristics of the integrated circuit device, the result of compilation including first data representative of parameters of the artificial neural network and second data representative of first instructions executable by the integrated circuit device to generate an output of the artificial neural network based on the first data and an input to the artificial neural network, and mapping, based on a specification indicating a capability of at least one different type of integrated circuit device, the result of the compilation generated using the description of the artificial neural network into at least one different compilation result for the at least one different type of integrated circuit to generate second instructions executable on the at least one different type of integrated circuit. ### Claim 11 (depends on 10) The computing device of claim 10, wherein the hardware characteristics of the integrated circuit device identify features, options, behaviors, performances, or latency of at least one processing unit of the integrated circuit device, or any combination thereof. ### Claim 12 (depends on 10) The computing device of claim 10, wherein the at least one microprocessor is further configured to generate test programs, load the test programs into the integrated circuit device for execution, and retrieve results of executing the test programs from the integrated circuit device; and wherein the hardware characteristics of the integrated circuit device are identified from the results. ### Claim 13 (depends on 12) The computing device of claim 12, wherein the at least one microprocessor is further configured to perform optimization of the result of compilation based on the hardware characteristics of the integrated circuit device. ### Claim 14 (depends on 13) The computing device of claim 13, further comprising the integrated circuit device coupled with the at least one microprocessor. ### Claim 15 (depends on 14) The computing device of claim 14, wherein the integrated circuit device comprises an integrated circuit die of a Field-Programmable Gate Array (FPGA) or Application Specific Integrated circuit (ASIC) implementing a Deep Learning Accelerator, the Deep Learning Accelerator comprising at least one processing unit configured to perform matrix operations. ### Claim 16 (depends on 15) The computing device of claim 15, wherein the at least one processing unit includes a matrix-matrix unit configured to operate on two matrix operands of an instruction; wherein the matrix-matrix unit includes a plurality of matrix-vector units configured to operate in parallel; wherein each of the plurality of matrix-vector units includes a plurality of vector-vector units configured to operate in parallel; and wherein each of the plurality of vector-vector units includes a plurality of multiply-accumulate units configured to operate in parallel. ### Claim 17 A non-transitory computer storage medium storing instructions which when executed by a computing device cause the computing device to perform a method, the method comprising: transmitting, by the computing device, one or more requests to an integrated circuit device to provide one or more responses indicative of hardware characteristics of the integrated circuit device in performing matrix computations; generating, by the computing device, a result of compilation from data representative of a description of an artificial neural network based at least in part on the hardware characteristics of the integrated circuit device, the result of compilation including first data representative of parameters of the artificial neural network and second data representative of first instructions executable by the integrated circuit device to generate an output of the artificial neural network based on the first data and an input to the artificial neural network; and mapping, based on a specification indicating a capability of at least one different type of integrated circuit device, the result of the compilation generated using the description of the artificial neural network into at least one different compilation result for the at least one different type of integrated circuit device to generate second instructions executable on the at least one different type of integrated circuit device. ### Claim 18 (depends on 17) The non-transitory computer storage medium of claim 17, wherein the hardware characteristics of the integrated circuit device identify features, options, behaviors, performances, or latency of at least one processing unit of the integrated circuit device, or any combination thereof. ### Claim 19 (depends on 18) The non-transitory computer storage medium of claim 18, wherein the method further comprises: generating test programs according to predetermined specifications of a plurality of hardware platforms of devices configured to perform matrix computations; loading the test programs into the integrated circuit device; instructing the integrated circuit device to execute the test programs; measuring performance levels of the integrated circuit device in executing instructions contained used in the test programs; and determining the hardware characteristics of the integrated circuit device based at least in part on the performance levels. ### Claim 20 (depends on 19) The non-transitory computer storage medium of claim 19, wherein the method further comprises: probing hardware options of a processing unit of the integrated circuit device based on results of executing the test programs in the integrated circuit device.