# US 2019/0318248 A1 · Automated feature generation, selection and hyperparameter tuning from structured data for supervised learning problems

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0318248 A1
- Document ID: US20190318248A1
- Status: Published application
- Kind: A1
- Application number: 16/011,651
- Filed: 2018-06-19
- Published: 2019-10-17
- Earliest claimed: 2018-04-13
- Assignees: NEC Laboratories Europe GmbH
- Applicants: NEC Laboratories Europe GmbH
- Inventors: Luis Moreira-Matias; Sourabh Sarvotham Parkala; Jihed Khiari; Diego Andres Espinoza Silva
- Web page: https://patentgrep.com/patent/US20190318248A1

## CPC

- G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
- G06N 20/20: Ensemble learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G06N 20/00: Machine learning
- G06N 3/126: Evolutionary algorithms, e.g. genetic algorithms or genetic programming
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods and systems for automating supervised learning tasks are provided. Feature generation in a feature space having a plurality of features using at least one predefined process for a plurality of data types is performed. A minimum set of relevant features are identified. The feature space is decreased using at least one filtering approach and the minimum set of relevant features. A Bayesian combinatorial optimization heuristic is devised to jointly identify a feature subset and a hyperparameter setting for a given query, a machine learning algorithm, and a dataset.

## Claims

### Claim 1

A method for automating supervised learning tasks comprising:
performing feature generation in a feature space having a plurality of features using at least one predefined process for a plurality of data types;
identifying a minimum set of relevant features;
decreasing the feature space using at least one filtering approach and the minimum set of relevant features; and
devising a Bayesian combinatorial optimization heuristic to jointly identify a feature subset and a hyperparameter setting for a given query, a machine learning algorithm, and a dataset.

### Claim 2 (depends on 1)

The method of claim 1 further comprising removing features using at least one of near zero variance, correlation analysis, lasso, and random forest.

### Claim 3 (depends on 1)

The method of claim 1 wherein identifying a minimum set of relevant features further comprises removing two features from the feature space.

### Claim 4 (depends on 1)

The method of claim 1 further comprising ranking features by importance.

### Claim 5 (depends on 1)

The method of claim 1 further comprising evaluating a generalization error for the hyperparameter setting.

### Claim 6 (depends on 5)

The method of claim 5 further comprising determining whether the generalization error has increased monotonically over a given number of iterations.

### Claim 7 (depends on 1)

The method of claim 1 further comprising recovering a relevant feature from a set of non-selected features.

### Claim 8 (depends on 1)

The method of claim 1 further comprising performing preprocessing on the dataset.

### Claim 9 (depends on 1)

The method of claim 1 further comprising outputting the Bayesian combinatorial optimization heuristic.

### Claim 10 (depends on 1)

The method of claim 1 further comprising refining the Bayesian combinatorial optimization heuristic by assessing it with the dataset.

### Claim 11

A configuration system comprising one or more processors which, alone or in combination, are configured to provide for performance of the following steps:
performing feature generation in a feature space having a plurality of features using at least one predefined process for a plurality of data types;
identifying a minimum set of relevant features;
decreasing the feature space using at least one filtering approach and the minimum set of relevant features; and
devising a Bayesian combinatorial optimization heuristic to jointly identify a feature subset and a hyperparameter setting for a given query, a machine learning algorithm, and a dataset.

### Claim 12 (depends on 11)

The system of claim 11 further comprising removing features using at least one of near zero variance, correlation analysis, lasso, and random forest.

### Claim 13 (depends on 11)

The system of claim 11 wherein identifying a minimum set of relevant features further comprises removing two features from the feature space.

### Claim 14 (depends on 11)

The system of claim 11 further comprising ranking features by importance.

### Claim 15 (depends on 11)

The system of claim 11 further comprising evaluating a generalization error for the hyperparameter setting.