# US 12,217,085 B2 · Electronic device including accelerators of heterogeneous hardware types for scheduling operation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,085 B2 - Document ID: US12217085B2 - Status: Granted - Kind: B2 - Application number: 17/402,848 - Filed: 2021-08-16 - Published: 2022-09-08 - Granted: 2025-02-04 - Earliest claimed: 2021-03-03 - Assignees: Samsung Electronics Co., Ltd.; Seoul National University R&Db Foundation - Applicants: Samsung Electronics Co., Ltd.; Seoul National University R&DB Foundation - Inventors: Seung Wook Lee; Jangwoo Kim; Pyeongsu Park - Web page: https://patentgrep.com/patent/US12217085B2 ## CPC - G06F 9/4881: {Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues} - G06F 9/541: {via adapters, e.g. between incompatible applications} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An electronic device includes: a host processor configured to control an operation of the electronic device; accelerators of heterogeneous hardware types configured to exchange data with each other through direct communication; and a control unit configured to convert a command received from the host processor, based on a type of each of the accelerators and transfer a result of the converting to a corresponding accelerator among the accelerators. ## Claims ### Claim 1 An electronic device, comprising: accelerators of heterogeneous hardware types, comprising different interfaces, configured to exchange data with each other through direct communication; and a controller, without interference by a host processor and a software stack, configured to: perform an interface translation on a command corresponding to an operation received from the host processor, based on a type of each of the accelerators and an interface difference between the accelerators; transfer a result of the interface translation on the command to an accelerator corresponding to the command among the accelerators; and the accelerator corresponding to the command executes the transferred result of the interface translation on the command, wherein the accelerators and the controller are distinguished from the host processor that controls the electronic device, wherein among the accelerators there is an accelerator with a first interface type and an accelerator with a second interface type that is different than the first interface type which directly communicate with each other without interference by the host processor and the software stack. ### Claim 2 (depends on 1) The electronic device of claim 1, further comprising: an interconnect switch configured to manage the direct communication between the accelerators without interference by the host processor and the software stack. ### Claim 3 (depends on 1) The electronic device of claim 1, wherein the controller is further configured to: divide the operation into a data receiving operation, a data processing operation, and a data transferring operation; divide each of the data receiving operation, the data processing operation, and the data transferring operation into respective sub-operations; and perform scheduling of each of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation for one or more of the accelerators based on a dependency between the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation and characteristics of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation. ### Claim 4 (depends on 3) The electronic device of claim 3, wherein the controller is further configured to: perform scheduling of each of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation for one or more of the accelerators based on complexity, repeatability, execution time of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation, or any combination of any two or more of complexity, repeatability, and execution time of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation. ### Claim 5 (depends on 1) The electronic device of claim 1, wherein the accelerators comprise any combination of any two or more of an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a graphics processing unit (GPU), a central processing unit (CPU), a neural processing unit (NPU), a tensor processing unit (TPU), and a digital signal processor (DSP). ### Claim 6 (depends on 1) The electronic device of claim 1, wherein the software stack comprises at least one of a driver, a library, and an application. ### Claim 7 An electronic device, comprising: accelerators of heterogeneous hardware types with different interfaces configured to exchange data with each other through direct communication; and an interconnect switch, configured to manage the direct communication between the accelerators without interference by a host processor and a software stack; a controller configured to, without interference by the host processor and the software stack, convert a command corresponding to an operation received from the host processor based on a type of each of the accelerators and transfer a result of the conversion of the command to an accelerator corresponding to the command among the accelerators; the accelerator corresponding to the command executes the transferred result of the conversion of the command; wherein the accelerators and the interconnect switch are distinguished from the host processor that controls the electronic device, and wherein among the accelerators there is an accelerator with a first interface type and an accelerator with a second interface type that is different than the first interface type which directly communicate with each other without interference by the host processor and the software stack. ### Claim 8 (depends on 7) The electronic device of claim 7, wherein the controller is further configured to: perform an interface translation on the command received from the host processor based on an interface difference between the accelerators; and transfer a result of the interface translation on the command to the accelerator corresponding to the command. ### Claim 9 (depends on 7) The electronic device of claim 7, wherein the controller is further configured to: divide the operation into a data receiving operation, a data processing operation, and a data transferring operation; divide each of the data receiving operation, the data processing operation, and the data transferring operation into respective sub-operations; and perform scheduling of each of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation for one or more of the accelerators based on a dependency between the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation and characteristics of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation. ### Claim 10 (depends on 9) The electronic device of claim 9, wherein the controller is further configured to: perform scheduling of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation for one or more of the accelerators based on complexity, repeatability, execution time of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation, or any combination of any two or more of complexity, repeatability, and execution time of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation. ### Claim 11 (depends on 7) The electronic device of claim 7, wherein the accelerators comprise any combination of any two or more of an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a graphics processing unit (GPU), and a central processing unit (CPU). ### Claim 12 (depends on 7) The electronic device of claim 7, wherein the software stack comprises at least one of a driver, a library, and an application. ### Claim 13 An electronic device, comprising: accelerators of heterogeneous hardware types with different interfaces configured to exchange data with each other through direct communication; and a controller, without interference by a host processor that controls the electronic device and without interference by a software stack, configured to: divide an operation into a data receiving operation, a data processing operation, and a data transferring operation; divide each of the data receiving operation, the data processing operation, and the data transferring operation into respective sub-operations; perform scheduling of each of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation for one or more of the accelerators based on a dependency between the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation and characteristics of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation; and the one or more of the accelerators execute the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation, wherein among the accelerators there is an accelerator with a first interface type and an accelerator with a second interface type that is different than the first interface type which directly communicate with each other without interference by the host processor and the software stack. ### Claim 14 (depends on 13) The electronic device of claim 13, wherein the controller is further configured to: convert a command corresponding to the operation received from the host processor based on a type of each of the accelerators and transfer a result of the conversion of the command to an accelerator corresponding to the command among the accelerators. ### Claim 15 (depends on 13) The electronic device of claim 13, further comprising: an interconnect switch configured to manage the direct communication between the accelerators without interference by the host processor and the software stack. ### Claim 16 (depends on 13) The electronic device of claim 13, wherein, as the controller divides each of the data receiving operation, the data processing operation, and the data transferring operation into the respective sub-operations, an accelerator, among the accelerators, is configured to: receive first sub-data at a first time point; process the first sub-data at a second time point; and process second sub-data, receive third sub-data, and transfer the processed first sub-data at a third time point. ### Claim 17 (depends on 13) The electronic device of claim 13, wherein the performance of the scheduling of each of the sub-operations of the data receiving operation, the sub-operations of the data processing operation, and the sub-operations of the data transferring operation is based on characteristics of the one or more of the accelerators. ### Claim 18 (depends on 13) The electronic device of claim 13, wherein the software stack comprises at least one of a driver, a library, and an application.