# US 12,001,382 B2 · Methods, apparatus, and articles of manufacture to generate command lists to be offloaded to accelerator circuitry > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,382 B2 - Document ID: US12001382B2 - Status: Granted - Kind: B2 - Application number: 17/559,556 - Filed: 2021-12-22 - Published: 2022-04-14 - Granted: 2024-06-04 - Earliest claimed: 2021-12-22 - Assignees: Intel Corporation - Applicants: Intel Corporation - Inventors: Celine Lee; Niranjan Hasabnis; Paul Petersen; Justin Gottschlich; Ramesh Peri - Web page: https://patentgrep.com/patent/US12001382B2 ## CPC - G06F 15/80: comprising an array of processing units with common control, e.g. single instruction multiple data processors (G06F15/82 takes precedence {; for correlation function computation G06F17/15}) - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06F 13/4068: {Electrical coupling} - G06F 15/7867: {with reconfigurable architecture} - G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] - G06N 3/045: Combinations of networks - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06N 3/092: Reinforcement learning - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N 20/00: Machine learning - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract Methods, apparatus, systems, and articles of manufacture are disclosed to generate command lists to be offloaded to accelerator circuitry. An example apparatus includes kernel duration model circuitry to predict a duration of execution of a first kernel based on a first source location, a first name, a first property of a first argument, or an occupancy of the first kernel. The example apparatus includes subsequent kernel model circuitry to predict a tuple and a dependency of a second kernel based on a second source location, a second name, a second property of a second argument, or a time of submission of the previous kernel. The example apparatus includes reinforcement learning model circuitry to determine whether to bundle the first kernel into a command list based on the duration of execution of the first kernel, the tuple of the second kernel, or the dependency of the second kernel. ## Claims ### Claim 1 An apparatus to generate command lists to be offloaded to accelerator circuitry, the apparatus comprising: offload control circuitry to obtain a first tuple of a first kernel to be offloaded to the accelerator circuitry; and processor circuitry including one or more of: at least one of a central processor unit (CPU), a graphics processor unit (GPU), or a digital signal processor (DSP), the at least one of the CPU, the GPU, or the DSP having control circuitry to control data movement within the processor circuitry, arithmetic and logic circuitry to perform one or more first operations corresponding to instructions, and one or more registers to store a first result of the one or more first operations, the instructions in the apparatus; Field Programmable Gate Array (FPGA) circuitry, the FPGA circuitry including first logic gate circuitry, a plurality of configurable interconnections, and storage circuitry, the first logic gate circuitry and interconnections to perform one or more second operations, the storage circuitry to store a second result of the one or more second operations; or Application Specific Integrated Circuitry (ASIC) including second logic gate circuitry to perform one or more third operations; the processor circuitry to perform at least one of the first operations, the second operations, or the third operations to instantiate: kernel duration model circuitry to predict a duration of execution of the first kernel based on at least one of a first source location of the first kernel, a first name of the first kernel, a first property of a first argument of the first kernel, or an occupancy of the first kernel; subsequent kernel model circuitry to predict a second tuple of a second kernel and a dependency of the second kernel based on at least one of a second source location of a previous kernel, a second name of the previous kernel, a second property of a second argument of the previous kernel, or a time of submission of the previous kernel; and reinforcement learning model circuitry to determine whether to bundle the first kernel into a command list based on at least one of (a) the duration of execution of the first kernel, (b) the second tuple of the second kernel, or (c) the dependency of the second kernel. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the processor circuitry is to perform at least one of the first operations, the second operations, or the third operations to instantiate kernel occupancy determination circuitry to determine the occupancy of the first kernel based on the first property of the first argument of the first kernel. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the time of submission of the previous kernel is a first time of submission, and the first tuple includes (a) the first property of the first argument of the first kernel, (b) a second time of submission of the first kernel, and (c) at least one of the first source location of the first kernel or the first name of the first kernel. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the processor circuitry is to perform at least one of the first operations, the second operations, or the third operations to instantiate the reinforcement learning (RL) model circuitry to adjust the RL model based on a time cost associated with bundling of the first kernel into the command list. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the processor circuitry to perform at least one of the first operations, the second operations, or the third operations to instantiate the kernel duration model circuitry to: determine a difference between an actual duration of execution of the first kernel and the predicted duration of execution of the first kernel from the kernel duration model circuitry; and adjust the kernel duration model based on the difference. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the processor circuitry to perform at least one of the first operations, the second operations, or the third operations to instantiate the subsequent kernel model circuitry to: determine a first difference between an actual tuple of the second kernel and the predicted second tuple of the second kernel from the subsequent kernel model circuitry; determine a second difference between an actual dependency of the second kernel and the predicted dependency of the second kernel from the subsequent kernel model circuitry; and adjust the subsequent kernel model based on at least one of the first difference or the second difference. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the GPU is a first GPU, and the accelerator circuitry includes a second GPU. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the time of submission of the previous kernel is a first time of submission, and, to predict the second tuple of the second kernel, the subsequent kernel model circuitry is to predict at least one of a third source location of the second kernel, a third name of the second kernel, a third property of a third argument of the second kernel, or a second time of submission of the second kernel. ### Claim 9 A non-transitory computer readable medium comprising machine readable instructions which, when executed, cause programmable processor circuitry to: predict, with a kernel duration model, a duration of execution of a first kernel based on at least one of a first source location of the first kernel, a first name of the first kernel, a first property of a first argument of the first kernel, or an occupancy of the first kernel, the first kernel to be offloaded to accelerator circuitry; predict, with a subsequent kernel model, a tuple of a second kernel and a dependency of the second kernel based on at least one of a second source location of a previous kernel, a second name of the previous kernel, a second property of a second argument of the previous kernel, or a time of submission of the previous kernel; and determine, with a reinforcement learning model, whether to bundle the first kernel into a command list based on at least one of (a) the duration of execution of the first kernel, (b) the tuple of the second kernel, or (c) the dependency of the second kernel. ### Claim 10 (depends on 9) The non-transitory computer readable medium of claim 9, wherein the instructions cause the programmable processor circuitry to determine the occupancy of the first kernel based on the first property of the first argument of the first kernel. ### Claim 11 (depends on 9) The non-transitory computer readable medium of claim 9, wherein the tuple of the second kernel is a first tuple, the time of submission of the previous kernel is a first time of submission, and a second tuple of the first kernel includes (a) the first property of the first argument of the first kernel, (b) a second time of submission of the first kernel, and (c) at least one of the first source location of the first kernel or the first name of the first kernel. ### Claim 12 (depends on 9) The non-transitory computer readable medium of claim 9, wherein the instructions cause the programmable circuitry to adjust the reinforcement learning model based on a time cost associated with bundling of the first kernel into the command list. ### Claim 13 (depends on 9) The non-transitory computer readable medium of claim 9, wherein the instructions cause the programmable circuitry to: determine a difference between an actual duration of execution of the first kernel and the predicted duration of execution of the first kernel; and adjust the kernel duration model based on the difference. ### Claim 14 (depends on 9) The non-transitory computer readable medium of claim 9, wherein the instructions cause the programmable circuitry to: determine a first difference between an actual tuple of the second kernel and the predicted tuple of the second kernel; determine a second difference between an actual dependency of the second kernel and the predicted dependency of the second kernel; and adjust the subsequent kernel model based on at least one of the first difference or the second difference. ### Claim 15 (depends on 9) The non-transitory computer readable medium of claim 9, wherein the time of submission of the previous kernel is a first time of submission, and to predict the tuple of the second kernel the instructions cause the programmable circuitry to predict at least one of a third source location of the second kernel, a third name of the second kernel, a third property of a third argument of the second kernel, or a second time of submission of the second kernel. ### Claim 16 An apparatus to generate command lists to be offloaded to accelerator circuitry, the apparatus comprising: at least one memory; machine readable instructions; programmable circuitry to at least one of instantiate or execute the machine readable instructions to: predict, with a kernel duration model, a duration of execution of a first kernel based on at least one of a first source location of the first kernel, a first name of the first kernel, a first property of a first argument of the first kernel, or an occupancy of the first kernel; predict, with a subsequent kernel model, a tuple of a second kernel and a dependency of the second kernel based on at least one of a second source location of a previous kernel, a second name of the previous kernel, a second property of a second argument of the previous kernel, or a time of submission of the previous kernel; and determine, with a reinforcement learning model, whether to bundle the first kernel into a command list based on at least one of (a) the duration of execution of the first kernel, (b) the tuple of the second kernel, or (c) the dependency of the second kernel. ### Claim 17 (depends on 16) The apparatus of claim 16, wherein the programmable circuitry is to determine the occupancy of the first kernel based on the first property of the first argument of the first kernel. ### Claim 18 (depends on 16) The apparatus of claim 16, wherein the tuple of the second kernel is a first tuple, the time of submission of the previous kernel is a first time of submission, and a second tuple of the first kernel includes (a) the first property of the first argument of the first kernel, (b) a second time of submission of the first kernel, and (c) at least one of the first source location of the first kernel or the first name of the first kernel. ### Claim 19 (depends on 16) The apparatus of claim 16, wherein the programmable circuitry is to adjust the reinforcement learning model based on a time cost associated with bundling of the first kernel into the command list. ### Claim 20 (depends on 16) The apparatus of claim 16, wherein the programmable circuitry is to: determine a difference between an actual duration of execution of the first kernel and the predicted duration of execution of the first kernel; and adjust the kernel duration model based on the difference. ### Claim 21 (depends on 16) The apparatus of claim 16, wherein the programmable circuitry is to: determine a first difference between an actual tuple of the second kernel and the predicted tuple of the second kernel; determine a second difference between an actual dependency of the second kernel and the predicted dependency of the second kernel; and adjust the subsequent kernel model based on at least one of the first difference or the second difference. ### Claim 22 (depends on 16) The apparatus of claim 16, wherein the time of submission of the previous kernel is a first time of submission, and to predict the tuple of the second kernel the programmable circuitry is to predict at least one of a third source location of the second kernel, a third name of the second kernel, a third property of a third argument of the second kernel, or a second time of submission of the second kernel. ### Claim 23 A method for generating command lists to be offloaded to accelerator circuitry, the method comprising: predicting, with a kernel duration model, a duration of execution of a first kernel based on at least one of a first source location of the first kernel, a first name of the first kernel, a first property of a first argument of the first kernel, or an occupancy of the first kernel; predicting, with a subsequent kernel model, a tuple of a second kernel and a dependency of the second kernel based on at least one of a second source location of a previous kernel, a second name of the previous kernel, a second property of a second argument of the previous kernel, or a time of submission of the previous kernel; and determining, with a reinforcement learning model, whether to bundle the first kernel into a command list based on at least one of (a) the duration of execution of the first kernel, (b) the tuple of the second kernel, or (c) the dependency of the second kernel. ### Claim 24 (depends on 23) The method of claim 23, wherein the tuple of the second kernel is a first tuple, the time of submission of the previous kernel is a first time of submission, and a second tuple of the first kernel includes (a) the first property of the first argument of the first kernel, (b) a second time of submission of the first kernel, and (c) at least one of the first source location of the first kernel or the first name of the first kernel. ### Claim 25 (depends on 23) The method of claim 23, wherein the time of submission of the previous kernel is a first time of submission, the second kernel is a subsequent kernel that is to be offloaded to the accelerator circuitry, and predicting the tuple of the subsequent kernel includes predicting at least one of a third source location of the subsequent kernel, a third name of the subsequent kernel, a third property of a third argument of the subsequent kernel, or a second time of submission of the subsequent kernel.