Multi-task feature selection neural networks
AssigneeGeneral Electric Company
InventorsHariharan Ravishankar, Bharath Ram Sundar, Prasad Sudhakar, Rahul Venkataramani, Vivek Vaidya
Life of this patent
Estimating the term…
Abstract
The present approach relates to feature ranking within deep neural networks in a multi-task and/or multi-label setting. Approaches are described to identify features that are task-specific as well as features that are shared across multiple tasks. In addition to facilitating interpretability, the selected subset of features can be used to make efficient models leading to better stability & regularization along with reduced compute and memory.
Claims20
A feature ranking neural network, comprising:
- an input layer;
- a broadcast layer encoding a plurality of weights, wherein the plurality of weights comprise a task specific weight for each task of a plurality of tasks for which the neural network is trained;
- two or more separate and parallel branches of the feature ranking neural network, wherein the branches are configured to receive a set of filtered inputs from the broadcast layer based on the weights encoded by the broadcast layer, wherein each branch corresponds to a different task, and wherein an orthogonalization technique is applied to the plurality of weights to identify at least one feature shared between the two or more separate and parallel branches, and wherein the set of filtered inputs represent electrocardiogram data values;
- and
- an output layer downstream from the branches, wherein the output layer is configured to provide an output of the feature ranking neural network, wherein the output indicates a presence or an absence of an inferior myocardial infarction.
A feature ranking method, comprising:
- providing as a training data set a plurality of input samples to an input layer of a neural network, wherein each input sample is characterized by one or more features, wherein the neural network comprises a plurality of layers, and wherein one of the plurality of layers is a broadcast layer comprising a respective weight for each task of a plurality of tasks;
- processing the input samples to train the respective weight for each task for each of the one or more features, and wherein the input samples represent electrocardiogram data values;
- and
- for one or more tasks of the plurality of tasks, identifying one or more features as being relevant to a respective task based on the respective weights of the one or more features with respect to the respective tasks, wherein an orthogonalization technique is applied to the respective weights to identify at least one feature shared between at least two of the plurality of layers, wherein an output based on the one or more features indicates a presence or an absence of an inferior myocardial infarction.
A method for generating a reduced feature set model, comprising:
- acquiring one or more weights associated with a broadcast layer of a trained neural network, wherein each weight is associated with a respective feature and task combination, wherein the one or more weights correspond to input samples representing electrocardiogram data values;
- identifying one or more task-specific features for a given task based on the weights;
- and
- generating a model based on the one or more task-specific features, wherein the one or more task-specific features is a subset of a larger feature set for which the trained neural network was trained, and wherein an orthogonalization technique is applied to the one or more weights to identify at least one feature shared between the two or more layers of the trained neural network, and wherein an output from the model indicates a presence or an absence of an inferior myocardial infarction.
Dependencies are inferred from the claim text.