Back to results
US 12,423,158 B2Granted
System and method for provisioning of artificial intelligence accelerator (AIA) resources
AssigneeSolidRun Ltd.
InventorsAtai Ziv, Rabeeh Khoury, Kossay Omary, Avi Levy
Life of this patent
Estimating the term…
Abstract
A neural processing unit (NPU) includes a network interface controller (NIC) connected to at least one client device over a network fabric in a manner such that the network interface controller can receive instructions from at least one client device. The NPU further includes an artificial intelligence accelerator (AIA) connected to the network interface controller, the AIA being arranged to receive instructions from the NIC and execute the received instructions.
Claims20
A system for provisioning a plurality of neural processing units (NPUs), comprising:
- a network interface controller (NIC) connected to at least one client device over a network fabric, wherein the NIC and the network fabric are configured to provide connectivity between the at least one client device and a plurality of NPUs, wherein the NIC is further configured to receive instructions from the at least one client device over the network fabric;
- an artificial intelligence accelerator (AIA) directly connected to the NIC without any intervening processor, wherein the AIA is configured to receive the instructions from the NIC and execute the received instructions;
- and
- a memory containing instructions which when executed by a processing circuitry of the system configures the system to:
- receive a request for provisioning an NPU of the plurality of NPUs;
- receive a signal from an available NPU of the plurality of NPUs;
- provision the available NPU to the client device based on the request;
- and
- provision, when the available NPU does not provide sufficient resources to satisfy the request, an additional NPU from an NPU tier which matches a client device tier such that an output of the available NPU serves as an input for the additional NPU, wherein the client device tier is based on a status of the client device.
6 more elements
An orchestrator comprising:
- a processing circuitry;
- and
- a memory containing instructions that, when executed by the processing circuitry, configure the orchestrator to:
- receive a request for a neural processing unit (NPU) from a client device;
- receive a signal from an available NPU;
- provision the available NPU to the client device based on the request;
- and
- when the available NPU does not provide sufficient resources to satisfy the request, provision an additional NPU from an NPU tier which matches a client device tier such that an output of the available NPU serves as an input for the additional NPU, wherein the client device tier is based on a status of the client device, and, wherein the NPU comprises:
- a network interface controller (NIC) connected to at least one client device including the client device over a network fabric, wherein the NIC is configured to receive instructions from the at least one client device over the network fabric;
- and
- an artificial intelligence accelerator (AIA) directly connected to the NIC without any intervening processor, wherein the AIA is configured to receive the instructions from the NIC and execute the received instructions.
8 more elements
A method comprising:
- receiving a request for a neural processing unit (NPU) from a client device;
- receiving a signal from an available NPU;
- provisioning the available NPU to the client device based on the request;
- and
- when the available NPU does not provide sufficient resources to satisfy the request, provisioning an additional NPU from an NPU tier which matches a client device tier such that an output of the available NPU serves as an input for the additional NPU, wherein the client device tier is based on a status of the client device, and wherein the NPU comprises:
- a network interface controller (NIC) connected to at least one client device including the client device over a network fabric, wherein the NIC is configured to receive instructions from the at least one client device over the network fabric;
- and
- an artificial intelligence accelerator (AIA) directly connected to the NIC without any intervening processor, wherein the AIA is configured to receive the instructions from the NIC and execute the received instructions.
5 more elements
Dependencies are inferred from the claim text.