US 2022/0036123 A1Published application

Machine learning model scaling system with energy efficient network data transfer for power aware hardware

AssigneeIntel Corporation

InventorsDaniel J. Cummings, Juan Pablo Munoz, Souvik Kundu, Sharath Nittur Sridhar, Maciej Szankin

Life of this patent

Estimating the term…

Loading patent…

Abstract

The present disclosure is related to machine learning model swap (MLMS) framework for that selects and interchanges machine learning (ML) models in an energy and communication efficient way while adapting the ML models to real time changes in system constraints. The MLMS framework includes an ML model search strategy that can flexibly adapt ML models for a wide variety of compute system and/or environmental changes. Energy and communication efficiency is achieved by using a similarity-based ML model selection process, which selects a replacement ML model that has the most overlap in pre-trained parameters from a currently deployed ML model to minimize memory write operation overhead. Other embodiments may be described and/or claimed.

Claims27

Dependent claims

An apparatus employed as machine learning model swap (MLMS) system, the apparatus comprising:

interface circuitry to obtain context information of a hardware (HW) platform on which a machine learning (ML) model is currently deployed;
and
processor circuitry connected to the interface circuitry, the processor circuitry is to:
detect a status change of the HW platform based on the context information of the HW platform and based on inference results generated by the currently deployed ML model,
trigger an MLMS engine to perform an MLMS, wherein the MLMS involves replacement of the currently deployed ML model with a replacement ML model, and the replacement ML model is optimized for running on the HW platform when the HW platform is operating under one or more constraints indicated by the context information such that the replacement ML model is to run on the HW platform under the one or more constraints with one or more improved performance metrics in comparison to performance of the currently deployed ML model being run on the HW platform under the one or more constraints, and
provision the replacement ML model to the HW platform.

3 more elements

One or more non-transitory computer readable media (NTCRM) comprising instructions for a machine learning model swap (MLMS) strategy, wherein execution of the instructions by one or more processors of a compute node is to cause the compute node to:

obtain context information of a hardware (HW) platform on which a machine learning (ML) model is currently deployed;
obtain inference results generated by the currently deployed ML model;
detect operational changes of the HW platform based on the context information and the inference results;
in response to the detected operational changes, determine a replacement ML model to replace the currently deployed ML model, wherein the replacement ML model is optimized for operating on the HW platform with the detected operational changes such that the replacement ML model is configured to run on the HW platform under the detected operational changes with one or more improved performance metrics in comparison to performance of the currently deployed ML model being run on the HW platform under the detected operational changes;
and
provision the determined replacement ML model to the HW platform.

3 more elements

Dependencies are inferred from the claim text.

Facts

Application: 17/506,161Filed 20 Oct 2021
Published: 3 Feb 2022
Term ends: Not estimatedThe source records don’t support an estimate.

Classification17 codes

G06F 18/285
Analysing: selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
G06V 10/82
Using pattern recognition or machine learning: using neural networks
G06F 18/217
Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation: validation; Performance…
G06N 3/045
Architecture, e.g. interconnection topology: combinations of networks

Family

US 12,417,260 B2Related patent
US 2022/0036123 A1This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key