Artificial intelligence inference architecture with hardware acceleration
AssigneeIntel Corporation
InventorsFrancesc Guim Bernat, Ned M. Smith
Life of this patent
Estimating the term…
Abstract
Various systems and methods of artificial intelligence (AI) processing using hardware acceleration within edge computing settings are described herein. In an example, processing performed at an edge computing device includes: obtaining a request for an AI operation using an AI model; identifying, based on the request, an AI hardware platform for execution of an instance of the AI model; and causing execution of the AI model instance using the AI hardware platform. Further operations to analyze input data, perform an inference operation with the AI model, and coordinate selection and operation of the hardware platform for execution of the AI model, is also described.
Claims23
A computing device to implement an inferencing-as-a-service deployment in a communication network, the computing device comprising:
- a network interface card (NIC);
- and
- processing circuitry coupled to the NIC, the processing circuitry configured to:
- access a plurality of artificial intelligence (AI) models and metadata information of the AI models;
- decode a query for an AI workload received via the NIC, the query associated with input data;
- select an AI model of the plurality of AI models based on the query and the metadata information;
- decode the query to access a binary for an instance of the AI model of the plurality of AI models;
- select a hardware accelerator from a plurality of hardware accelerators based on the query, the hardware accelerator being configurable to be in multiple hardware platforms and/or multiple appliances;
- and
- cause execution of the binary for the instance of the AI model using the hardware accelerator to process the input data.
At least one non-transitory machine-readable storage medium comprising instructions stored thereupon, which when executed by processing circuitry of a computing node operable to implement an inferencing-as-a-service deployment in a communication network, cause the processing circuitry to perform operations comprising:
- accessing a plurality of artificial intelligence (AI) models and metadata information of the AI models;
- decoding a query for an AI workload, the query associated with input data;
- selecting an AI model of the plurality of AI models based on the query and the metadata information;
- decoding the query to access a binary for an instance of the AI model of the plurality of AI models;
- selecting a hardware accelerator from a plurality of hardware accelerators based on the query, the hardware accelerator being configurable to be in multiple hardware platforms and/or multiple appliances;
- and
- causing execution of the binary for the instance of the AI model using the hardware accelerator to process the input data.
A method comprising:
- performing by at least one hardware processor:
- accessing a plurality of artificial intelligence (AI) models and metadata information of the AI models;
- decoding a query for an AI workload, the query associated with input data;
- selecting an AI model of the plurality of AI models based on the query and the metadata information;
- decoding the query to access a binary for an instance of the AI model of the plurality of AI models;
- selecting a hardware accelerator from a plurality of hardware accelerators based on the query, the hardware accelerator being configurable to be in multiple hardware platforms and/or multiple appliances;
- and
- causing execution of the binary for the instance of the AI model using the hardware accelerator to process the input data.
Dependencies are inferred from the claim text.