# US 10,372,428 B1 · Dynamic computational acceleration using a heterogeneous hardware infrastructure > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,372,428 B1 - Document ID: US10372428B1 - Status: Granted - Kind: B1 - Application number: 15/989,064 - Filed: 2018-05-24 - Granted: 2019-08-06 - Earliest claimed: 2015-06-16 - Assignees: Architecture Technology Corporation - Applicants: Architecture Technology Corporation - Inventors: Judson Powers; Noel E. Schmidt - Web page: https://patentgrep.com/patent/US10372428B1 ## CPC - G06F 8/41: Compilation - G06F 8/447: {Target code generation} - G06F 8/451: {Code distribution (considering CPU load at run-time G06F9/505; load rebalancing G06F9/5083)} - G06F 9/455: Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines - G06F 9/45504: {Abstract machines for program code execution, e.g. Java virtual machine [JVM], interpreters or emulators} - G06F 9/45516: {Runtime code conversion or optimisation} - G06F 9/46: Multiprogramming arrangements - G06F 9/4843: {by program, e.g. task dispatcher, supervisor, operating system} - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE ## Abstract An example method includes, during execution of a software application in a computing system comprising a plurality of processing units, identifying platform-independent instructions that are configured to perform at least one computational task, wherein the plurality of processing units comprises a heterogeneous group, and wherein the platform-independent instructions have a format that is not specific to any particular processing unit in the plurality of processing units, determining one or more scheduling criteria that are associated with the platform-independent instructions, and selecting, from the heterogeneous group of processing units and based on the scheduling criteria, a processing unit to perform the at least one computational task. The example method further includes converting the platform-independent instructions into platform-dependent instructions, wherein the platform-dependent instructions have a format that is specific to the selected processing unit, and executing, by the selected processing unit, the platform-dependent instructions to perform the at least one computational task. ## Claims ### Claim 1 A method comprising: in a distributed computing environment comprising a plurality of distributed computing systems that are communicatively coupled to one another via at least one network, identifying, during execution of a software application, platform-independent instructions that are configured to perform at least one computational task associated with execution of the software application, wherein the plurality of computing systems includes a heterogeneous group of specialized processing units, wherein the platform-independent instructions have a platform-independent format that is not specific to any particular processing unit of the heterogeneous group of specialized processing units, wherein the software application is associated with an application executable previously compiled from application source code using a first compiler, wherein the software application is further associated with the platform-independent instructions that, prior to execution of the software application, were compiled from computational source code using a second compiler, and wherein the platform-independent instructions are not native to or executable by any particular processing unit of the heterogeneous group of specialized processing units; determining real-time performance information to collect and store during performance of the at least one computational task by the heterogeneous group of specialized processing units, wherein the real-time performance information indicates an amount of one or more constrained resources utilized by one or more of the specialized processing units while performing the at least one computational task during execution of platform-dependent instructions that are specific to each respective specialized processing unit; after identifying the platform-independent instructions, determining one or more scheduling criteria that are associated with the platform-independent instructions, wherein the one or more scheduling criteria are based at least in part on the real-time performance information previously collected while performing the at least one computational task by the one or more of the specialized processing units during prior execution of platform-dependent instructions specific to each respective specialized processing unit; selecting, from the heterogeneous group of specialized processing units and based on the one or more scheduling criteria, a specialized processing unit to perform the at least one computational task, wherein selecting the specialized processing unit comprises selecting one of the heterogeneous group of specialized processing units that utilizes a relatively fewer amount of the one or more constrained resources, with respect to one or more other specialized processing units in the group, when performing the at least one computational task during execution of platform-dependent instructions specific to the selected specialized processing unit; during execution of the software application, converting the platform-independent instructions into the platform-dependent instructions that are specific to the selected specialized processing unit, wherein the application executable includes references, provided by one or more libraries, to the platform-independent instructions that are resolved at runtime during execution of the software application, and wherein the platform-dependent instructions comprise optimized native code executable by the selected specialized processing unit; and providing the platform-dependent instructions to the selected specialized processing unit for execution in order to perform the at least one computational task. ### Claim 2 (depends on 1) The method of claim 1, wherein the one or more constrained resources include memory utilized by one or more of the specialized processing units while performing the at least one computational task, wherein the real-time performance information indicates the amount of memory utilized by one or more of the specialized processing units while performing the at least one computational task, and wherein the selected specialized processing unit utilizes a relatively smaller amount of the memory, with respect to one or more other specialized processing units in the group, when performing the at least one computational task during execution of the platform-dependent instructions specific to the selected specialized processing unit. ### Claim 3 (depends on 1) The method of claim 1, wherein the one or more scheduling criteria are further based at least in part on a respective number of the platform-independent instructions that are executable on one or more of the heterogeneous group of specialized processing units. ### Claim 4 (depends on 1) The method of claim 1, wherein the one or more scheduling criteria are further based at least in part on one or more of a size of the platform-independent instructions, or a number of times that the platform-independent instructions are invoked during execution of the software application. ### Claim 5 (depends on 1) The method of claim 1, wherein the one or more scheduling criteria are further based at least in part on an amount of time to previously convert the platform-independent instructions into platform-dependent instructions specific to one or more of the heterogeneous group of specialized processing units. ### Claim 6 (depends on 1) The method of claim 1, wherein the heterogeneous group of specialized processing units comprises at least a graphics processing unit and a field-programmable gate array. ### Claim 7 (depends on 1) The method of claim 1, wherein the platform-independent instructions have a device-independent intermediate representation comprising Low Level Virtual Machine (LLVM) bitcode. ### Claim 8 (depends on 1) The method of claim 1, wherein converting the platform-independent instructions into the platform-dependent instructions specific to the selected specialized processing unit comprises compiling the platform-independent instructions into the platform-dependent instructions that are executable by the selected specialized processing unit. ### Claim 9 (depends on 1) The method of claim 1, further comprising: storing collected performance information that is associated with performance of the at least one computational task during execution of the platform-dependent instructions by the selected specialized processing unit. ### Claim 10 A computing system, comprising: at least one processing unit; and a non-transitory computer-readable storage medium storing instructions that, when executed, cause the at least one processing unit to: in a distributed computing environment comprising a plurality of distributed computing systems that are communicatively coupled to one another via at least one network, identify, during execution of a software application, platform-independent instructions that are configured to perform at least one computational task associated with execution of the software application, wherein the plurality of computing systems includes a heterogeneous group of specialized processing units, wherein the platform-independent instructions have a platform-independent format that is not specific to any particular processing unit of the heterogeneous group of specialized processing units, wherein the software application is associated with an application executable previously compiled from application source code using a first compiler, wherein the software application is further associated with the platform-independent instructions that, prior to execution of the software application, were compiled from computational source code using a second compiler, and wherein the platform-independent instructions are not native to or executable by any particular processing unit of the heterogeneous group of specialized processing units; determine real-time performance information to collect and store during performance of the at least one computational task by the heterogeneous group of specialized processing units, wherein the real-time performance information indicates an amount of one or more constrained resources utilized by one or more of the specialized processing units while performing the at least one computational task during execution of platform-dependent instructions that are specific to each respective specialized processing unit; after identifying the platform-independent instructions, determine one or more scheduling criteria that are associated with the platform-independent instructions, wherein the one or more scheduling criteria are based at least in part on the real-time performance information previously collected while performing the at least one computational task by the one or more of the specialized processing units during prior execution of platform-dependent instructions specific to each respective specialized processing unit; select, from the heterogeneous group of specialized processing units and based on the one or more scheduling criteria, a specialized processing unit to perform the at least one computational task, wherein selecting the specialized processing unit comprises selecting one of the heterogeneous group of specialized processing units that utilizes a relatively fewer amount of the one or more constrained resources, with respect to one or more other specialized processing units in the group, when performing the at least one computational task during execution of platform-dependent instructions specific to the selected specialized processing unit; during execution of the software application, convert the platform-independent instructions into the platform-dependent instructions that are specific to the selected specialized processing unit, wherein the application executable includes references, provided by one or more libraries, to the platform-independent instructions that are resolved at runtime during execution of the software application, and wherein the platform-dependent instructions comprise optimized native code executable by the selected specialized processing unit; and provide the platform-dependent instructions to the selected specialized processing unit for execution in order to perform the at least one computational task. ### Claim 11 (depends on 10) The computing system of claim 10, wherein the one or more constrained resources include memory utilized by one or more of the specialized processing units while performing the at least one computational task, wherein the real-time performance information indicates the amount of memory utilized by one or more of the specialized processing units while performing the at least one computational task, and wherein the selected specialized processing unit utilizes a relatively smaller amount of the memory, with respect to one or more other specialized processing units in the group, when performing the at least one computational task during execution of the platform-dependent instructions specific to the selected specialized processing unit. ### Claim 12 (depends on 10) The computing system of claim 10, wherein the one or more scheduling criteria are further based at least in part on a respective number of the platform-independent instructions that are executable on one or more of the heterogeneous group of specialized processing units. ### Claim 13 (depends on 10) The computing system of claim 10, wherein the one or more scheduling criteria are further based at least in part on one or more of a size of the platform-independent instructions, or a number of times that the platform-independent instructions are invoked during execution of the software application. ### Claim 14 (depends on 10) The computing system of claim 10, wherein the one or more scheduling criteria are further based at least in part on an amount of time to previously convert the platform-independent instructions into platform-dependent instructions specific to one or more of the heterogeneous group of specialized processing units. ### Claim 15 (depends on 10) The computing system of claim 10, wherein the heterogeneous group of specialized processing units comprises at least a graphics processing unit and a field-programmable gate array. ### Claim 16 (depends on 10) The computing system of claim 10, wherein the platform-independent instructions have a device-independent intermediate representation comprising Low Level Virtual Machine (LLVM) bitcode. ### Claim 17 (depends on 10) The computing system of claim 10, wherein the instructions further cause the at least one processing unit to: store collected performance information that is associated with performance of the at least one computational task during execution of the platform-dependent instructions by the selected specialized processing unit. ### Claim 18 A non-transitory computer-readable storage medium comprising instructions that, when executed, cause a computing system to perform operations comprising: in a distributed computing environment comprising a plurality of distributed computing systems that are communicatively coupled to one another via at least one network, identifying, during execution of a software application, platform-independent instructions that are configured to perform at least one computational task associated with execution of the software application, wherein the plurality of computing systems includes a heterogeneous group of specialized processing units, wherein the platform-independent instructions have a platform-independent format that is not specific to any particular processing unit of the heterogeneous group of specialized processing units, wherein the software application is associated with an application executable previously compiled from application source code using a first compiler, wherein the software application is further associated with the platform-independent instructions that, prior to execution of the software application, were compiled from computational source code using a second compiler, and wherein the platform-independent instructions are not native to or executable by any particular processing unit of the heterogeneous group of specialized processing units; determining real-time performance information to collect and store during performance of the at least one computational task by the heterogeneous group of specialized processing units, wherein the real-time performance information indicates an amount of one or more constrained resources utilized by one or more of the specialized processing units while performing the at least one computational task during execution of platform-dependent instructions that are specific to each respective specialized processing unit; after identifying the platform-independent instructions, determining one or more scheduling criteria that are associated with the platform-independent instructions, wherein the one or more scheduling criteria are based at least in part on the real-time performance information previously collected while performing the at least one computational task by the one or more of the specialized processing units during prior execution of platform-dependent instructions specific to each respective specialized processing unit; selecting, from the heterogeneous group of specialized processing units and based on the one or more scheduling criteria, a specialized processing unit to perform the at least one computational task, wherein selecting the specialized processing unit comprises selecting one of the heterogeneous group of specialized processing units that utilizes a relatively fewer amount of the one or more constrained resources, with respect to one or more other specialized processing units in the group, when performing the at least one computational task during execution of platform-dependent instructions specific to the selected specialized processing unit; during execution of the software application, converting the platform-independent instructions into the platform-dependent instructions that are specific to the selected specialized processing unit, wherein the application executable includes references, provided by one or more libraries, to the platform-independent instructions that are resolved at runtime during execution of the software application, and wherein the platform-dependent instructions comprise optimized native code executable by the selected specialized processing unit; and providing the platform-dependent instructions to the selected specialized processing unit for execution in order to perform the at least one computational task. ### Claim 19 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein the one or more constrained resources include memory utilized by one or more of the specialized processing units while performing the at least one computational task, wherein the real-time performance information indicates the amount of memory utilized by one or more of the specialized processing units while performing the at least one computational task, and wherein the selected specialized processing unit utilizes a relatively smaller amount of the memory, with respect to one or more other specialized processing units in the group, when performing the at least one computational task during execution of the platform-dependent instructions specific to the selected specialized processing unit. ### Claim 20 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein the one or more scheduling criteria are further based at least in part on one or more of a size of the platform-independent instructions, or a number of times that the platform-independent instructions are invoked during execution of the software application. ### Claim 21 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein the one or more scheduling criteria are further based at least in part on an amount of time to previously convert the platform-independent instructions into platform-dependent instructions specific to one or more of the heterogeneous group of specialized processing units. ### Claim 22 (depends on 18) The non-transitory computer-readable storage medium of claim 18, wherein the platform-independent instructions have a device-independent intermediate representation comprising Low Level Virtual Machine (LLVM) bitcode.