# US 12,761,987 B2 · Interface for multiple processors > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,761,987 B2 - Document ID: US12761987B2 - Status: Granted - Kind: B2 - Application number: 17/189,090 - Filed: 2021-03-01 - Published: 2022-09-01 - Granted: 2026-10-13 - Earliest claimed: 2021-03-01 - Assignees: NVIDIA Corporation - Applicants: NVIDIA Corporation - Inventors: Lopamudra Kundu; Elena Agostini - Web page: https://patentgrep.com/patent/US12761987B2 ## CPC - G06F 9/544: {Buffers; Shared memory; Pipes} - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Apparatuses, systems, and techniques to interface with an accelerator. In at least one embodiment, an application provides workloads to a logical device, and the logical device distributes the workloads across a plurality of accelerators. ## Claims ### Claim 1 A non-transitory machine-readable medium having stored thereon executable instructions, which if performed by one or more processors, cause the one or more processors to, in response to an application programming interface (API) call, at least: identify a workflow submitted via the API call, the workflow comprising two or more workloads to be performed sequentially, each workload corresponding to a profile describing resource requirements and a type of accelerator to perform a corresponding workload; select at least two accelerators from a plurality of accelerators to perform the two or more workloads based at least in part on respective profiles of the two or more workloads; cause a first workload of the two or more workloads to be scheduled to be performed by a first accelerator of the at least two accelerators; and wherein the first accelerator is to use information in the workflow submitted via the API call to forward intermediate results computed by the first accelerator in performing the first workload to a second accelerator of the at least two accelerators. ### Claim 2 (depends on 1) The non-transitory machine-readable medium of claim 1, wherein performance of the executable instructions further causes the one or more processors to, in response to the API call: remove a workflow generated by an application from a queue, the workflow having a plurality of workloads; cause performance of a first workload of the plurality of workloads on the first accelerator; and cause performance of a second workload of the plurality of workloads on the second accelerator. ### Claim 3 (depends on 2) The non-transitory machine-readable medium of claim 2, wherein: each workload in the plurality of workloads has an associated acceleration profile provided by the application; and the API call directs an individual workload of the plurality of workloads to a particular accelerator based at least in part on an acceleration profile associated with the individual workload. ### Claim 4 (depends on 3) The non-transitory machine-readable medium of claim 3, wherein: the first workload has a first acceleration profile and the second workload has a second acceleration profile; and the first acceleration profile is different from the second acceleration profile. ### Claim 5 (depends on 2) The non-transitory machine-readable medium of claim 2, wherein the API obtains the workflow from the queue in a single dequeue operation. ### Claim 6 (depends on 1) The non-transitory machine-readable medium of claim 1, wherein the information is transferred from the first accelerator of the at least two accelerators to the second accelerator of the at least two accelerators using direct memory access. ### Claim 7 (depends on 1) The non-transitory machine-readable medium of claim 1, wherein the API implements a logical device that provides interfaces to both the first accelerator and the second accelerator of the at least two accelerators. ### Claim 8 (depends on 1) The non-transitory machine-readable medium of claim 1, wherein the first accelerator or the second accelerator of the at least two accelerators is a field programmable gate array, an application specific integrated circuit, a digital signal processor, a graphics processing unit, or a central processing unit. ### Claim 9 (depends on 1) The non-transitory machine-readable medium of claim 1, wherein the intermediate results further include instructions to be performed by the second accelerator of the at least two accelerators. ### Claim 10 (depends on 1) The non-transitory machine-readable medium of claim 1, wherein the two or more workloads in the workflow are distributed to the first accelerator, and the first accelerator uses execution sequence information in the workflow to forward one or more other workloads of the two or more workloads to the second accelerator, along with the intermediate results computed by the first accelerator in performing the first workload. ### Claim 11 A computer system comprising one or more processors and machine-readable media to store executable instructions that, as a result of being performed by the one or more processors, cause the computer system to, in response to an application programming interface (API) call: identify a workflow submitted via the API call, the workflow comprising two or more workloads to be performed sequentially, each workload corresponding to a profile describing resource requirements and type of accelerator to perform a corresponding workload; select at least two accelerators from a plurality of accelerators to perform the two or more workloads based at least in part on respective profiles of the two or more workloads; and cause a first workload of the two or more workloads to be scheduled to be performed by a first accelerator of the at least two accelerators; wherein the first accelerator is to use information in the workflow submitted via the API call to forward intermediate results computed by the first accelerator in performing the first workload to a second accelerator of the at least two accelerators. ### Claim 12 (depends on 11) The computer system of claim 11, wherein, in response to the API call, the one or more processors are further to: remove, from a queue of workflows, a plurality of workloads in the form of a single workflow submitted by an application; perform a first workload of the plurality of workloads on the first accelerator; and cause the second accelerator to perform a second workload of the plurality of workloads. ### Claim 13 (depends on 12) The computer system of claim 12, wherein each workload in the plurality of workloads has an associated acceleration profile that identifies a capability of an accelerator required to perform the associated workload. ### Claim 14 (depends on 13) The computer system of claim 13, wherein a first workload in the plurality of workloads and a second workload in the plurality of workloads have different acceleration profiles. ### Claim 15 (depends on 12) The computer system of claim 12, wherein the first accelerator and the second accelerator perform portions of a workflow serially. ### Claim 16 (depends on 13) The computer system of claim 13, wherein the API causes an individual workload of the plurality of workloads to be performed by a particular accelerator based at least in part on an acceleration profile associated with the individual workload. ### Claim 17 (depends on 11) The computer system of claim 11, wherein the API implements a single logical device that is capable of dispatching workloads to both the first accelerator and the second accelerator. ### Claim 18 (depends on 11) The computer system of claim 11, wherein the first accelerator and the second accelerator of the at least two accelerators perform portions of a workflow in parallel. ### Claim 19 (depends on 11) The computer system of claim 11, wherein the information further includes executable instructions to be performed by the second accelerator. ### Claim 20 A computer-implemented method, comprising: receiving an application programming interface (API) call; and in response to receipt of the API call: identifying a workflow submitted via the API call, the workflow comprising two or more workloads to be performed sequentially, each workload corresponding to a profile describing resource requirements and a type of accelerator to perform a corresponding workload; selecting at least two accelerators from a plurality of accelerators to perform the two or more workloads based at least in part on respective profiles of the two or more workloads; and causing a first workload of the two or more workloads to be scheduled to be performed by a first accelerator of the at least two accelerators; wherein the first accelerator is to use information in the workflow submitted via the API call to forward intermediate results computed by the first accelerator in performing the first workload to a second accelerator of the at least two accelerators. ### Claim 21 (depends on 20) The computer-implemented method of claim 20, further comprising: in response to the API call: removing a workflow generated by an application from a queue, the workflow having a plurality of workloads; causing a first workload of the plurality of workloads to be performed on the first accelerator; and causing a second workload of the plurality of workloads to be performed on the second accelerator. ### Claim 22 (depends on 21) The computer-implemented method of claim 21, wherein each workload in the plurality of workloads has an associated acceleration profile describing a characteristic of an accelerator able to perform the workload. ### Claim 23 (depends on 22) The computer-implemented method of claim 22, wherein: the plurality of workloads includes a first workload with a first acceleration profile and a second workload with a second acceleration profile; and the first acceleration profile is different from the second acceleration profile. ### Claim 24 (depends on 21) The computer-implemented method of claim 21, wherein the API obtains the workflow from the queue in a single dequeue operation. ### Claim 25 (depends on 22) The computer-implemented method of claim 22, wherein the API directs an individual workload of the plurality of workloads to a particular accelerator based at least in part on an acceleration profile associated with the individual workload. ### Claim 26 (depends on 20) The computer-implemented method of claim 20, wherein the API includes a logical device that distributes workloads to the first accelerator and the second accelerator. ### Claim 27 (depends on 26) The computer-implemented method of claim 26, wherein the first workload and a second workload are performed serially by the first accelerator and the second accelerator. ### Claim 28 (depends on 20) The computer-implemented method of claim 20, wherein the information further includes instructions to be performed by the second accelerator. ### Claim 29 One or more processors, comprising: circuitry to, in response to an application programming interface (API) call: identify a workflow submitted via the API call, the workflow comprising two or more workloads to be performed sequentially, each workload corresponding to a profile describing resource requirements and a type of accelerator to perform a corresponding workload; select at least two accelerators from a plurality of accelerators to perform the two or more workloads based at least in part on respective profiles of the two or more workloads; and cause a first workload of the two or more workloads to be scheduled to be performed by a first accelerator of the at least two accelerators; wherein the first accelerator is to use information in the workflow submitted via the API call to forward intermediate results computed by the first accelerator in performing the first workload to a second accelerator of the at least two accelerators. ### Claim 30 (depends on 29) The one or more processors of claim 29, wherein the circuitry is to, in response to the API call: remove a workflow generated by an application from a queue, the workflow having a plurality of workloads; cause a first workload of the plurality of workloads to be performed on the first accelerator; and cause a second workload of the plurality of workloads to be performed on the second accelerator. ### Claim 31 (depends on 30) The one or more processors of claim 30, wherein each workload in the plurality of workloads is associated with an acceleration profile accessible to the API. ### Claim 32 (depends on 31) The one or more processors of claim 31, wherein: individual workloads in the plurality of workloads have different acceleration profiles; and the different acceleration profiles cause the plurality of workloads to be performed by different types of accelerators. ### Claim 33 (depends on 30) The one or more processors of claim 30, wherein the information further includes an intermediate result produced by the first accelerator. ### Claim 34 (depends on 30) The one or more processors of claim 30, wherein an application adds an entire workflow to the queue in a single enqueue operation. ### Claim 35 (depends on 30) The one or more processors of claim 30, wherein the first accelerator is a virtual processor. ### Claim 36 (depends on 31) The one or more processors of claim 31, wherein the API directs an individual workload of the plurality of workloads to a particular accelerator based at least in part on an acceleration profile associated with the individual workload. ### Claim 37 (depends on 29) The one or more processors of claim 29, wherein the API implements a logical device that interfaces with the first accelerator and the second accelerator.