# US 2022/0035877 A1 · Hardware-aware machine learning model search mechanisms

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0035877 A1
- Document ID: US20220035877A1
- Status: Published application
- Kind: A1
- Application number: 17/504,996
- Filed: 2021-10-19
- Published: 2022-02-03
- Earliest claimed: 2021-10-19
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Sharath Nittur Sridhar; Anthony Sarah
- Web page: https://patentgrep.com/patent/US20220035877A1

## CPC

- G06F 16/953: Querying, e.g. by the use of web search engines
- G06N 3/086: using evolutionary algorithms, e.g. genetic algorithms or genetic programming
- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

The present disclosure is related to framework for automatically and efficiently finding machine learning (ML) architectures that generalize well across multiple artificial intelligence (AI) and/or ML domains, AI/ML tasks, and datasets. The ML architecture search framework accepts a list of tasks and corresponding datasets as inputs, and may also include relevancy scores/weights for each item in the input. A combined performance metric is generated, where this combined performance metric quantifies the performance of the ML architecture across all the specified AI/ML domains, AI/ML tasks, and datasets. The system then performs a multi-objective ML architecture search with the combined performance metric, along with hardware-specific performance metrics as the objectives. Other embodiments may be described and/or claimed.

## Claims

### Claim 1

An apparatus for providing a machine learning architecture search (MLAS) engine, the apparatus comprising:
an input mapper configured to:
generate a task list from a set of specified tasks obtained from a client device, and
identify individual datasets of a set of datasets to be generalized datasets or task-specific datasets, the task-specific datasets are datasets that can only be used for one task in the task list and the generalized datasets are datasets that can be used for more than one task in the task list;
a multi-task batch generator configured to generate one or more data batches from the individual datasets based on identified ones of the task-specific datasets and the generalized datasets;
a metric generator configured to:
determine a performance metric of each task in the task list, and
generate a combined performance metric (CPM) based on the determined performance metric of each task;
a multi-objective MLAS (MOMLAS) function configured to determine a set of generalized machine learning architectures (GMLAs) and corresponding performance metrics based on optimization of a multi-objective function, wherein each GMLA of the set of GMLAs is a machine learning (ML) architecture that can be used for more than one ML task and using more than one type of dataset; and
an MLAS interface configured to provide the set of GMLAs and the corresponding performance metrics for deployment on a hardware platform.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the CPM and one or more platform-based performance metrics are objectives of the multi-objective function and the set of GMLAs are a result of application of the multi-objective function.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein the input mapper is further configured to:
determine task-specific weights for corresponding tasks in the task list, wherein the task-specific weights indicate a preference of the corresponding tasks with respect to other tasks in the task list, and the task-specific weights are coefficients that correspond to respective ones of the performance metrics in the multi-objective function.

### Claim 4 (depends on 3)

The apparatus of claim 3, further comprising:
interface circuitry configured to obtain an ML configuration from a client device, the ML configuration including the set of specified tasks, the set of datasets, the task-specific weights, and the platform-based performance metrics.

### Claim 5 (depends on 3)

The apparatus of claim 3, further comprising a system application programming interface (API) configured to:
pass the task list and the task-specific weights to the metric generator; and
generate the individual datasets from the task-specific datasets and the generalized datasets such that the individual datasets correspond to a respective task in the task list.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the input mapper is configured to identify the individual datasets to be generalized datasets or task-specific datasets using a predetermined or configured ruleset, the ruleset including one or more rules for mapping the individual datasets to a set of generalized datasets or a set of task-specific datasets.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the multi-task batch generator is further configured to:
apply one or more pre-processing techniques on the individual datasets based on a predefined or configured ruleset, wherein the ruleset defines rules for translating the individual datasets into a format that is readable by the MOMLAS function.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein, to determine the performance metric of each task in the task list, the metric generator is configured to:
select a performance metric for each task based on a predefined or configured ruleset.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein, to generate the CPM, the metric generator is configured to:
calculate the CPM from the selected performance metric for each task using a predefined or configured statistical function.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the CPM is calculated using a configured statistical function, and the configured statistical function is included in an ML configuration provided by a client device.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the multi-objective function includes one or more of a predetermined multi-objective evolutionary algorithm (MOEA), a predetermined Neural Architecture Search (NAS) algorithm, or a configured multi-objective function included in an ML configuration provided by a client device.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the MOEA is Strength Pareto Evolutionary Algorithm 2 (SPEA-2) or Non-dominated Sorting Genetic Algorithm-II (NSGA-II).

### Claim 13 (depends on 1)

The apparatus of claim 1, wherein the set of GMLAs comprises a set of Pareto optimal solutions of the multi-objective function.

### Claim 14 (depends on 1)

The apparatus of claim 1, wherein the MOMLAS function is configured to:
operate each GMLA in the set of GMLAs using a test dataset; and
measure the corresponding performance metrics based on the operation of respective GMLAs in the set of GMLAs.

### Claim 15 (depends on 1)

The apparatus of claim 1, wherein the MOMLAS function is configured to:
predict the corresponding performance metrics of each GMLA using one or more proxy functions, wherein the one or more proxy functions include one or more of associative arrays, mapping functions, dictionaries, hash tables, look-up tables (LUTs), linked lists, ML classifiers, parameter counting, computational throughput metrics, Jacobian covariance functions, saliency pruning functions, channel pruning functions, heuristic functions, and hyper-heuristic functions.

### Claim 16

One or more non-transitory computer readable media (NTCRM) comprising instructions for operating a machine learning architecture search interface (MLASI), wherein execution of the instructions by one or more processors of a compute node is to cause the compute node to:
serve the MLASI to a client device; and
in response to receipt of an individual machine learning (ML) configuration from the client device via the MLASI:
determine a set of ML tasks and a set of datasets indicated by the individual ML configuration,
generate a task list from the set of ML tasks,
identify individual datasets of the set of datasets to be generalized datasets or task-specific datasets, the task-specific datasets being datasets that can only be used for one task in the task list and the generalized datasets being datasets that can be used for more than one task in the task list,
generate one or more data batches from individual datasets based on the task-specific datasets and the generalized datasets,
determine a performance metric of each task in the task list,
generate a combined performance metric (CPM) based on the determined performance metric of each task in the task list,
determine a set of generalized machine learning architectures (GMLAs) and corresponding performance metrics based on optimization of a multi-objective function, wherein each GMLA of the set of GMLAs is a machine learning (ML) architecture that can be used for more than one ML task and using more than one type of dataset, and wherein the set of GMLAs comprises a set of Pareto optimal solutions of the multi-objective function, and
indicate, via the MLASI, the set of GMLAs and the corresponding performance metrics for deployment on a hardware platform.

### Claim 17 (depends on 16)

The one or more NTCRM of claim 16, wherein the CPM and one or more platform-based performance metrics are objectives of the multi-objective function and the one or more GMLAs that are represented as decision variables of the multi-objective function.

### Claim 18 (depends on 17)

The one or more NTCRM of claim 17, wherein execution of the instructions is to further cause the compute node to:
determine task-specific weights for corresponding tasks in the task list, wherein the task-specific weights indicate a preference of the corresponding tasks with respect to other tasks in the task list, and the task-specific weights are coefficients that correspond to respective ones of the decision variables in the multi-objective function.

### Claim 19 (depends on 18)

The one or more NTCRM of claim 18, wherein the individual ML configuration includes the set of specified tasks, the set of datasets, the task-specific weights, and the platform-based performance metrics.

### Claim 20 (depends on 18)

The one or more NTCRM of claim 18, wherein execution of the instructions is to further cause the compute node to:
generate the individual datasets from the task-specific datasets and the generalized datasets such that the individual datasets correspond to a respective task in the task list.

### Claim 21 (depends on 16)

The one or more NTCRM of claim 16, wherein execution of the instructions is to further cause the compute node to:
identify the individual datasets to be generalized datasets or task-specific datasets using a predetermined or configured ruleset, the ruleset including one or more rules for mapping the individual datasets to a set of generalized datasets or a set of task-specific datasets.

### Claim 22 (depends on 16)

The one or more NTCRM of claim 16, wherein execution of the instructions is to further cause the compute node to:
apply one or more pre-processing techniques on the individual datasets based on a predefined or configured ruleset, wherein the ruleset defines rules for translating the individual datasets into a format that is readable by the MOMLAS function.

### Claim 23 (depends on 16)

The one or more NTCRM of claim 16, wherein, to determine the performance metric of each task in the task list, execution of the instructions is to further cause the compute node to:
select a performance metric for each task based on a predefined or configured ruleset.

### Claim 24 (depends on 23)

The one or more NTCRM of claim 23, wherein, to generate the CPM, execution of the instructions is to further cause the compute node to:
calculate the CPM from the selected performance metric for each task using a predefined or configured statistical function.

### Claim 25 (depends on 16)

The one or more NTCRM of claim 16, wherein the multi-objective function includes one or more of a predetermined multi-objective evolutionary algorithm (MOEA), a predetermined Neural Architecture Search (NAS) algorithm, or a configured multi-objective function included in an ML configuration provided by a client device.
