# US 2025/0156749 A1 · Automatic model selection relating to accuracy and performance

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0156749 A1
- Document ID: US20250156749A1
- Status: Published application
- Kind: A1
- Application number: 18/508,547
- Filed: 2023-11-14
- Published: 2025-05-15
- Earliest claimed: 2023-11-14
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Yao Dong Liu; Bo Song; Dong Hai Yu; Jun Wang; Jiang Bo Kang; Xiao Ming Ma; Si Er Han
- Web page: https://patentgrep.com/patent/US20250156749A1

## CPC

- G06N 20/00: Machine learning
- G06N 20/20: Ensemble learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computer-implemented method for model selection is provided. The computer-implemented method includes building models for predicting characteristics of pipeline models, receiving user specifications for pipeline model performance, defining a metric for weighing the characteristics of the pipeline models, using the metric to iteratively reduce a number of the pipeline models capable of meeting the user specifications to a reduced number of the pipeline models, determining which one pipeline model of the reduced number of the pipeline models exhibits a best capability of meeting the user specifications and deploying the one pipeline model.

## Claims

### Claim 1

A computer-implemented method for model selection, the computer-implemented method comprising:
building models for predicting characteristics of pipeline models;
receiving user specifications for pipeline model performance;
defining a metric for weighing the characteristics of the pipeline models;
using the metric to iteratively reduce a number of the pipeline models capable of meeting the user specifications to a reduced number of the pipeline models;
determining which one pipeline model of the reduced number of the pipeline models exhibits a best capability of meeting the user specifications; and
deploying the one pipeline model.

### Claim 2 (depends on 1)

The computer-implemented method according to claim 1, wherein:
the models for predicting the characteristics of the pipeline models are trained using historical data and a portion of input data, and
the historical data comprises pipeline model selection data and an entirety of the input data.

### Claim 3 (depends on 1)

The computer-implemented method according to claim 1, wherein:
the models for predicting the characteristics of the pipeline models comprise first and second regression models and a classification model,
accuracy and training time characteristics are derivable from the first regression model, and
evaluation time and prediction time characteristics are derivable from the second regression model and the classification model.

### Claim 4 (depends on 1)

The computer-implemented method according to claim 1, wherein the user specifications comprise at least an expected maximum experiment time and an expected maximum prediction time.

### Claim 5 (depends on 1)

The computer-implemented method according to claim 1, wherein the using of the metric to iteratively reduce the number of the pipeline models to the reduced number of the pipeline models comprises:
checking a performance level of each of the pipeline models;
removing worst performing pipeline models; and
repeating the checking and the removing until the reduced number of the pipeline models is reached.

### Claim 6 (depends on 5)

The computer-implemented method according to claim 5, wherein a number of repeating phases exceeds a number of the worst performing pipeline models that are removed in each phase.

### Claim 7 (depends on 1)

The computer-implemented method according to claim 1, wherein the determining of which one pipeline model of the reduced number of the pipeline models exhibits the best capability of meeting the user specifications comprises:
confirming that each pipeline model of the reduced number of the pipeline models meets the user specifications; and
determining which pipeline model that has been confirmed to meet the user specifications exhibits the best capability.

### Claim 8

A computer program product for model selection, the computer program product comprising one or more computer readable storage media having computer readable program code collectively stored on the one or more computer readable storage media, the computer readable program code being executed by a processor of a computer system to cause the computer system to perform a method comprising:
building models for predicting characteristics of pipeline models;
receiving user specifications for pipeline model performance;
defining a metric for weighing the characteristics of the pipeline models;
using the metric to iteratively reduce a number of the pipeline models capable of meeting the user specifications to a reduced number of the pipeline models;
determining which one pipeline model of the reduced number of the pipeline models exhibits a best capability of meeting the user specifications; and
deploying the one pipeline model.

### Claim 9 (depends on 8)

The computer program product according to claim 8, wherein:
the models for predicting the characteristics of the pipeline models are trained using historical data and a portion of input data, and
the historical data comprises pipeline model selection data and an entirety of the input data.

### Claim 10 (depends on 8)

The computer program product according to claim 8, wherein:
the models for predicting the characteristics of the pipeline models comprise first and second regression models and a classification model,
accuracy and training time characteristics are derivable from the first regression model, and
evaluation time and prediction time characteristics are derivable from the second regression model and the classification model.

### Claim 11 (depends on 8)

The computer program product according to claim 8, wherein the user specifications comprise at least an expected maximum experiment time and an expected maximum prediction time.

### Claim 12 (depends on 8)

The computer program product according to claim 8, wherein the using of the metric to iteratively reduce the number of the pipeline models to the reduced number of the pipeline models comprises:
checking a performance level of each of the pipeline models;
removing worst performing pipeline models; and
repeating the checking and the removing until the reduced number of the pipeline models is reached.

### Claim 13 (depends on 12)

The computer program product according to claim 12, wherein a number of repeating phases exceeds a number of the worst performing pipeline models that are removed in each phase.

### Claim 14 (depends on 8)

The computer program product according to claim 8, wherein the determining of which one pipeline model of the reduced number of the pipeline models exhibits the best capability of meeting the user specifications comprises:
confirming that each pipeline model of the reduced number of the pipeline models meets the user specifications; and
determining which pipeline model that has been confirmed to meet the user specifications exhibits the best capability.

### Claim 15

A computing system comprising:
a processor;
a memory coupled to the processor; and
one or more computer readable storage media coupled to the processor, the one or more computer readable storage media collectively containing instructions that are executed by the processor via the memory to implement a method for model selection comprising:
building models for predicting characteristics of pipeline models;
receiving user specifications for pipeline model performance;
defining a metric for weighing the characteristics of the pipeline models;
using the metric to iteratively reduce a number of the pipeline models capable of meeting the user specifications to a reduced number of the pipeline models;
determining which one pipeline model of the reduced number of the pipeline models exhibits a best capability of meeting the user specifications; and
deploying the one pipeline model.

### Claim 16 (depends on 15)

The computing system according to claim 15, wherein:
the models for predicting the characteristics of the pipeline models are trained using historical data and a portion of input data, and
the historical data comprises pipeline model selection data and an entirety of the input data.

### Claim 17 (depends on 15)

The computing system according to claim 15, wherein:
the models for predicting the characteristics of the pipeline models comprise first and second regression models and a classification model,
accuracy and training time characteristics are derivable from the first regression model, and
evaluation time and prediction time characteristics are derivable from the second regression model and the classification model.

### Claim 18 (depends on 15)

The computing system according to claim 15, wherein the user specifications comprise at least an expected maximum experiment time and an expected maximum prediction time.

### Claim 19 (depends on 15)

The computing system according to claim 15, wherein the using of the metric to iteratively reduce the number of the pipeline models to the reduced number of the pipeline models comprises:
checking a performance level of each of the pipeline models;
removing worst performing pipeline models; and
repeating the checking and the removing until the reduced number of the pipeline models is reached,
wherein a number of repeating phases exceeds a number of the worst performing pipeline models that are removed in each phase.

### Claim 20 (depends on 15)

The computing system according to claim 15, wherein the determining of which one pipeline model of the reduced number of the pipeline models exhibits the best capability of meeting the user specifications comprises:
confirming that each pipeline model of the reduced number of the pipeline models meets the user specifications; and
determining which pipeline model that has been confirmed to meet the user specifications exhibits the best capability.
