US 2021/0241140 A1Published application

Hybrid methods and systems for feature selection

AssigneeThe Florida International University Board of Trustees

InventorsThejas Gubbi SADASHIVA, Sundararaj S. IYENGAR

Life of this patent

Estimating the term…

Abstract

Methods and systems for feature selection (FS) in machine learning (ML) are provided. The filter and wrapper methods can be combined to provide a hybrid FS method and system. The data can be clustered using mini-batch K-means clustering and ranked using normalized mutual information (NMI). The wrapper method can include using either a feature inclusion process or a least-ranked feature exclusion process that eliminates least-ranked features one by one from the ranking list.

Claims18

Dependent claims

A system for performing feature selection in machine learning, the system comprising:

a processor;
and
a machine-readable medium in operable communication with the processor and comprising instructions stored thereon that, when executed by the processor, perform the following steps:
receiving a dataset;
performing feature ranking on the dataset using a filter technique to obtain a ranking list of features;
and
performing feature selection on the ranking list using a wrapper technique,
the filter technique comprising clustering data of the dataset and then using normalized mutual information (NMI) as a metric for ranking to generate the ranking list of features,
the NMI calculated as follows:
NMI
⁡
(
Ω
,
S
)
=
MI
⁡
(
Ω
;
S
)
[
G
⁡
(
Ω
)
+
G
⁡
(
S
)
)
]
/
2
,
where
MI
⁡
(
Ω
;
S
)
=
∑
k
⁢
∑
j
⁢
P
⁡
(
d
k
⋂
s
j
)
⁢
log
⁢
P
⁡
(
d
k
⋂
s
j
)
P
⁡
(
d
k
)
⁢
P
⁡
(
s
j
)
,
⁢
G
⁡
(
Ω
)
=
-
∑
k
⁢
(
d
k
)
⁢
log
⁢
⁢
P
⁡
(
d
k
)
,
and
where Ω is a set of clusters, S is a set of classes, P(dk)=probability of data in cluster dk, P(sj)=probability of data in cluster sj, G(S) is an entropy of the set of classes, and P(dk∩sj)=probability of data being in a convergence of dk and sj,
the use of the filter technique and the wrapper technique improving a runtime of the processor, and
the wrapper technique comprising performing a feature inclusion process.

118 more elements

A method for performing feature selection in machine learning, the method comprising:

receiving, by a processor, a dataset;
performing, by the processor, feature ranking on the dataset using a filter technique to obtain a ranking list of features;
and
performing, by the processor, feature selection on the ranking list using a wrapper technique,
the filter technique comprising clustering data of the dataset and then using normalized mutual information (NMI) as a metric for ranking to generate the ranking list of features,
the NMI calculated as follows:
NMI
⁡
(
Ω
,
S
)
=
MI
⁡
(
Ω
;
S
)
[
G
⁡
(
Ω
)
+
G
⁡
(
S
)
)
]
/
2
,
where
MI
⁡
(
Ω
;
S
)
=
∑
k
⁢
∑
j
⁢
P
⁡
(
d
k
⋂
s
j
)
⁢
log
⁢
P
⁡
(
d
k
⋂
s
j
)
P
⁡
(
d
k
)
⁢
P
⁡
(
s
j
)
,
⁢
G
⁡
(
Ω
)
=
-
∑
k
⁢
(
d
k
)
⁢
log
⁢
⁢
P
⁡
(
d
k
)
,
and
where Ω is a set of clusters, S is a set of classes, P(dk)=probability of data in cluster dk, P(sj)=probability of data in cluster sj, G(S) is an entropy of the set of classes, and P(dk∩sj)=probability of data being in a convergence of dk and sj,
the use of the filter technique and the wrapper technique improving a runtime of the processor, and
the wrapper technique comprising performing a feature inclusion process.

115 more elements

A system for performing feature selection in machine learning, the system comprising:

a processor;
and
a machine-readable medium in operable communication with the processor and comprising instructions stored thereon that, when executed by the processor, perform the following steps:
receiving a dataset;
performing feature ranking on the dataset using a filter technique to obtain a ranking list of features;
and
performing feature selection on the ranking list using a wrapper technique,
the filter technique comprising clustering data of the dataset and then using normalized mutual information (NMI) as a metric for ranking to generate the ranking list of features,
the NMI calculated as follows:
NMI
⁡
(
Ω
,
S
)
=
MI
⁡
(
Ω
;
S
)
[
G
⁡
(
Ω
)
+
G
⁡
(
S
)
)
]
/
2
,
where
MI
⁡
(
Ω
;
S
)
=
∑
k
⁢
∑
j
⁢
P
⁡
(
d
k
⋂
s
j
)
⁢
log
⁢
P
⁡
(
d
k
⋂
s
j
)
P
⁡
(
d
k
)
⁢
P
⁡
(
s
j
)
,
⁢
G
⁡
(
Ω
)
=
-
∑
k
⁢
(
d
k
)
⁢
log
⁢
⁢
P
⁡
(
d
k
)
,
where Ω is a set of clusters, S is a set of classes, P(dk)=probability of data in cluster dk, P(sj)=probability of data in cluster sj, G(S) is an entropy of the set of classes, and P(dk∩sj)=probability of data being in a convergence of dk and sj,
the clustering of the data of the dataset comprising mini-batch K-means clustering,
the wrapper technique comprising removing redundant features that have a dependency on other features,
the wrapper technique comprising performing a feature inclusion process,
the use of the filter technique and the wrapper technique improving a runtime of the processor, and
the feature inclusion process comprising performing Algorithm 1:
Algorithm 1:
input:
set of ranked features S = {f0, f1, f2, . . . fm}, where m = total number of features,
obtained from the feature ranking phase, f0 is the highest ranked feature and fm is the
least ranked feature
output:
provides the selected set of features
initialization :
1:
Lst = S[0] prev = 0, where prev represents a previous accuracy of a model
2:
for k = 0 to m−1 do
3:
x_tst = x_tst [ Lst ]
4:
x_tr = x_tr [ Lst ]
5:
train the model based on any classifier and store an accuracy on acc
6:
if acc > prev then
7:
if (k ≠ m − 1) then
8:
add S[ k + 1 ] into the Lst
9:
prev = acc
10:
else
11:
end if
12:
else
13:
remove S [ k ] object from the Lst
14:
if (k ≠ m − 1) then
15:
add S[ k + 1 ] to the Lst
16:
else
17:
end if
18:
end if
19:
end for.

166 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/782,730Filed 5 Feb 2020
Published: 5 Aug 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification8 codes

G06N 5/04
Computing arrangements using knowledge-based models: inference or reasoning models
G06F 16/215
Design, administration or maintenance of databases: improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or…
G06F 16/285
Relational databases: clustering or classification
G06N 7/01
Computing arrangements based on specific mathematical models: probabilistic graphical models, e.g. probabilistic networks

Family

No related US applications are recorded in the indexed filing history.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key

patentgrep

Loading patent…

Abstract

Facts

Application

16/782,730Filed 5 Feb 2020

Published

5 Aug 2021

Term ends

Not estimatedThe source records don’t support an estimate.

Classification8 codes

G06N 5/04
Computing arrangements using knowledge-based models: inference or reasoning models
G06F 16/215
Design, administration or maintenance of databases: improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or…
G06F 16/285
Relational databases: clustering or classification
G06N 7/01
Computing arrangements based on specific mathematical models: probabilistic graphical models, e.g. probabilistic networks