# US 9,436,526 B2 · Parallel runtime execution on multiple processors > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,526 B2 - Document ID: US9436526B2 - Status: Granted - Kind: B2 - Application number: 14/163,726 - Filed: 2014-01-24 - Published: 2014-07-17 - Granted: 2016-09-06 - Earliest claimed: 2007-04-11 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Aaftab Munshi; Jeremy Sandmel - Web page: https://patentgrep.com/patent/US9436526B2 ## CPC - G06F 9/445: Program loading or initiating (bootstrapping G06F9/4401; security arrangements for program loading or initiating G06F21/57) - G06F 8/41: Compilation - G06F 8/447: {Target code generation} - G06F 9/44542: {Retargetable} - G06F 9/4843: {by program, e.g. task dispatcher, supervisor, operating system} - G06F 9/5044: {considering hardware capabilities} - G06F 9/541: {via adapters, e.g. between incompatible applications} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method and an apparatus that schedule a plurality of executables in a schedule queue for execution in one or more physical compute devices such as CPUs or GPUs concurrently are described. One or more executables are compiled online from a source having an existing executable for a type of physical compute devices different from the one or more physical compute devices. Dependency relations among elements corresponding to scheduled executables are determined to select an executable to be executed by a plurality of threads concurrently in more than one of the physical compute devices. A thread initialized for executing an executable in a GPU of the physical compute devices are initialized for execution in another CPU of the physical compute devices if the GPU is busy with graphics processing threads. Sources and existing executables for an API function are stored in an API library to execute a plurality of executables in a plurality of physical compute devices, including the existing executables and online compiled executables from the sources. ## Claims ### Claim 1 A computer implemented method comprising: receiving, from a host application executing on a host processor, a request to create a kernel containing executable code corresponding to source code for a task in the host application; creating, by the host processor, the kernel for execution on a compute device specified by the host application, the compute device selected by the host application from one or more compute identifiers received by the host application in response to a request to identify compute devices that match a processing requirement for the task; receiving, from the host application, a request to execute the kernel; and queuing, by the host processor, the kernel on an execution queue associated with the selected compute device. ### Claim 2 (depends on 1) The computer implemented method of claim 1, wherein the selected compute device is associated with a context that defines the execution queue. ### Claim 3 (depends on 2) The computer implemented method of claim 2, wherein creating the kernel comprises compiling the source code for the task in accordance with the context. ### Claim 4 (depends on 1) The computer implemented method of claim 1, wherein the selected compute device comprises multiple physical processors and the task comprises a plurality of threads of execution that are processed concurrently on the multiple physical processors. ### Claim 5 (depends on 4) The computer implemented method of claim 4, wherein the host application defines a number of threads to be executed as a block on a physical processor. ### Claim 6 (depends on 1) The computer implemented method of claim 1 further comprising: receiving, from the host application, a request to allocate global memory for an input stream and an output stream for the kernel, the global memory coupled to the selected compute device. ### Claim 7 (depends on 6) The computer implemented method of claim 6, wherein the host application writes data to the input stream for execution by the kernel on the selected compute device and reads data from the output stream in response to the execution of the kernel on the selected compute device. ### Claim 8 (depends on 6) The computer implemented method of claim 6, wherein the input stream and the output stream each comprises one of an image stream and a variable stream. ### Claim 9 (depends on 1) The computer implemented method of claim 1, wherein the requests from the host application are processed by an interpreter during execution of the host application. ### Claim 10 A non-transitory computer readable storage medium storing instructions that when executed by a host processor cause the host processor to perform operations comprising: receiving, from a host application executing on the host processor, a request to create a kernel containing executable code corresponding to source code for a task in the host application; creating the kernel for execution on a compute device specified by the host application, the compute device selected by the host application from one or more compute identifiers received by the host application in response to a request to identify compute devices that match a processing requirement for the task; receiving, from the host application, a request to execute the kernel; and queuing the kernel on an execution queue associated with the selected compute device. ### Claim 11 (depends on 10) The non-transitory computer readable storage medium of claim 10, wherein the selected compute device is associated with a context that defines the execution queue. ### Claim 12 (depends on 11) The non-transitory computer readable storage medium of claim 11, wherein creating the kernel comprises compiling the source code for the task in accordance with the context. ### Claim 13 (depends on 10) The non-transitory computer readable storage medium of claim 10, wherein the selected compute device comprises multiple physical processors and the task comprises a plurality of threads of execution that are processed concurrently on the multiple physical processors. ### Claim 14 (depends on 13) The non-transitory computer readable storage medium of claim 13, wherein the host application defines a number of threads to be executed as a block on a physical processor. ### Claim 15 (depends on 10) The non-transitory computer readable storage medium of claim 10, wherein the instructions further cause the host processor to receive, from the host application, a request to allocate global memory for an input stream and an output stream for the kernel, the global memory coupled to the selected compute device. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the host application writes data to the input stream for execution by the kernel on the selected compute device and reads data from the output stream in response to the execution of the kernel on the selected compute device. ### Claim 17 (depends on 15) The non-transitory computer readable storage medium of claim 15, wherein the input stream and the output stream each comprises one of an image stream and a variable stream. ### Claim 18 (depends on 10) The non-transitory computer readable storage medium of claim 10, wherein the requests from the host application are processed by an interpreter during execution of the host application. ### Claim 19 A system comprising: host processor coupled to a memory through a bus; a plurality of computer devices coupled to the host processor through a bus; and a runtime layer process stored as instructions in the memory to cause the host processor to receive, from a host application executing on the host processor, a request to create a kernel containing executable code corresponding to source code for a task in the host application; create the kernel for execution on a compute device specified by the host application, the compute device selected by the host application from one or more compute identifiers received by the host application in response to a request to identify compute devices that match a processing requirement for the task; receive, from the host application, a request to execute the kernel; and queue the kernel on an execution queue associated with the selected compute device. ### Claim 20 (depends on 19) The system of claim 19, wherein the selected compute device is associated with a context that defines the execution queue. ### Claim 21 (depends on 20) The system of claim 20, wherein creating the kernel comprises compiling the source code for the task in accordance with the context. ### Claim 22 (depends on 19) The system of claim 19, wherein the selected compute device comprises multiple physical processors and the task comprises a plurality of threads of execution that are processed concurrently on the multiple physical processors. ### Claim 23 (depends on 22) The system of claim 22, wherein the host application defines a number of threads to be executed as a block on a physical processor. ### Claim 24 (depends on 19) The system of claim 19 further comprising a global memory coupled to the host processor and to the selected compute device and wherein the instructions further cause the host processor to receive, from the host application, a request to allocate global memory for an input stream and an output stream for the kernel. ### Claim 25 (depends on 24) The system of claim 24, wherein the host application writes data to the input stream for execution by the kernel on the selected compute device and reads data from the output stream in response to the execution of the kernel on the selected compute device. ### Claim 26 (depends on 24) The system of claim 24, wherein the input stream and the output stream each comprises one of an image stream and a variable stream. ### Claim 27 (depends on 19) The system of claim 19 further comprising an interpreter process stored as instructions in the memory to cause the host processor to interpret the requests from the host application during execution of the host application.