Benchmarking and segment-based prediction of artificial intelligence model inference characteristics on local devices
AssigneeDell Products L.P.
InventorsSpencer Bull, Tyler Cox, Jacob Mink, Srikanth Kondapi
Life of this patent
Estimating the term…
Abstract
An information handling system receives a request for an artificial intelligence service that is associated with an artificial intelligence model and retrieves a manifest associated with the artificial intelligence model. The system also determines a current system capacity of the information handling system and determines a workload segmentation according to the manifest based on the current system capacity of the information handling system.
Claims20
A method comprising:
- receiving, by an information handling system, a request for an artificial intelligence service that is associated with an artificial intelligence model;
- retrieving a manifest associated with the artificial intelligence model;
- determining a current system capacity of the information handling system;
- and
- determining a workload segmentation according to the manifest based on the current system capacity of the information handling system.
An information handling system, comprising:
- a processor;
- and
- a memory coupled to the processor, the memory having program instructions stored thereon that upon execution cause the processor to:
- receive a request for an artificial intelligence service that is associated with an artificial intelligence model;
- retrieve a manifest associated with the artificial intelligence model;
- determine a current system capacity of the information handling system;
- and
- determine a workload segmentation according to the manifest based on the current system capacity of the information handling system.
A non-transitory computer-readable medium to store instructions that are executable to perform operations comprising:
- receiving a request at an information handling system for an artificial intelligence service that is associated with an artificial intelligence model;
- retrieving a manifest associated with the artificial intelligence model;
- determining a current system capacity of the information handling system;
- and
- determining a workload segmentation according to the manifest based on the current system capacity of the information handling system.
Dependencies are inferred from the claim text.