# US 2023/0404461 A1 · Identifying cardiac abnormalities in multi-lead ecgs using hybrid neural network with fulcrum based data re-balancing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0404461 A1
- Document ID: US20230404461A1
- Status: Published application
- Kind: A1
- Application number: 18/329,855
- Filed: 2023-06-06
- Published: 2023-12-21
- Earliest claimed: 2022-06-17
- Assignees: Tata Consultancy Services Limited
- Applicants: Tata Consultancy Services Limited
- Inventors: VARSHA SHARMA; AYAN MUKHERJEE; MURALI PODUVAL; SUNDEEP KHANDELWAL; ANIRBAN DUTTA CHOUDHURY; CHIRAYATA BHATTACHARYYA
- Web page: https://patentgrep.com/patent/US20230404461A1

## CPC

- A61B 5/346: Analysis of electrocardiograms
- G06N 3/09: Supervised learning
- A61B 5/7203: {for noise prevention, reduction or removal}
- A61B 5/725: {using specific filters therefor, e.g. Kalman or adaptive filters (specific diagnostics methods using using bioelectric or biomagnetic signals A61B5/316)}
- A61B 5/7264: {Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems}
- A61B 5/7267: {involving training the classification device}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06T 1/00: General purpose image data processing
- G06N 3/048: Activation functions
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

State of art techniques hardly provide data balancing for multi-label multi-class data. Embodiments of the present disclosure provide a method and system for identifying cardiac abnormality in multi-lead ECGs using a Hybrid Neural Network (HNN) with fulcrum based data re-balancing for data comprising multiclass-multilabel cardiac abnormalities. The fulcrum based dataset re-balancing disclosed enables maintaining natural balance of the data, control the re-sample volume, and still support the lowly represented classes there by aiding proper training of the DL architecture. The HNN disclosed by the method utilizes a hybrid approach of pure CNN, a tuned-down version of ResNet, and a set of handcrafted features from a raw ECG signal that are concatenated prior to predicting the multiclass output for the ECG signal. The number of features is flexible and enables adding additional domain-specific features as needed.

## Claims

### Claim 1

A processor implemented method for identifying cardiac abnormalities in multi-lead electrocardiograms (ECGs), the method comprising:
receiving, via one or more hardware processors, a training data set comprising a plurality of ECG recordings with multilabel-multiclass cardiac abnormalities, wherein each of the plurality of ECGs are sampled at varying frequencies;
down sampling, via the one or more hardware processors, the plurality of ECG recordings with a single frequency pre-identified statistically;
pruning, via the one or more hardware processors, the down sampled plurality of ECG recordings to generate a pruned training dataset by eliminating one or more ECG recordings that fail to capture information related to at least one cardiac abnormality among a set of cardiac abnormalities identified as a set scoring classes;
slicing, via the one or more hardware processors, each ECG recording in the pruned training data set to generate a windowed dataset of overlapping windows;
applying, via the one or more hardware processors, a fulcrum-based data rebalancing technique on the windowed dataset to generate a rebalanced training dataset for the multiclass-multilabel cardiac abnormalities, the fulcrum-based data rebalancing technique comprising:
deriving a fulcrum point (Nfulcrum) of the windowed dataset from class frequency of each window in the windowed dataset based on one or more class labels from among the set of scoring classes assigned to each window, wherein one or more classes from among the set of scoring classes with class frequency above the fulcrum point are identified as majority classes, wherein classes with class frequency below the fulcrum point are identified as minority classes; and
performing a random sampling on the windowed dataset that is identified under minority classes to reach the fulcrum point, wherein random sampling provides a representation at least equal to the fulcrum point to the minority classes, wherein the majority classes are excluded from the random sampling; and
training, via the one or more hardware processors, a Hybrid Neural Network (HNN) architecture based DL model using the rebalanced training dataset to classify input ECG signals.

### Claim 2 (depends on 1)

The processor implemented method of claim 1, further comprising utilizing the trained DL model to classify an input ECG signal of a subject by predicting the abnormalities in the input ECG into one or more classes among the set of scoring classes.

### Claim 3 (depends on 2)

The processor implemented method of claim 2, wherein the HNN has three parallel layers comprising:
(i) a Convolutional Neural Network (CNN) layer receiving a multi-lead ECG input of the subject to learn a set of abstracted features, (ii) a Residual Network (ResNet) layer receiving a single lead ECG input to learn discriminating features associated with the heart activity of the subject, and (iii) a layer comprising a set of handcrafted features derived from raw data of the single lead ECG input, and wherein the CNN layer, the ResNet layer and the set of handcrafted features are concatenated to predict the cardiac abnormalities in the input ECG into one or more classes among the set of scoring classes.

### Claim 4 (depends on 1)

The processor implemented method of claim 1, wherein the fulcrum-based data rebalancing technique maintains natural balance of the windowed dataset, controls re-sample volume, and supports lowly represented classes for training of the HNN.

### Claim 5

A system for identifying cardiac abnormalities in multi-lead electrocardiograms (ECGs), the system comprising:
a memory storing instructions;
one or more Input/Output (I/O) interfaces; and
one or more hardware processors coupled to the memory via the one or more I/O interfaces, wherein the one or more hardware processors are configured by the instructions to:
receive a training data set comprising a plurality of ECG recordings with multilabel-multiclass cardiac abnormalities, wherein each of the plurality of ECGs are sampled at varying frequencies;
down sample the plurality of ECG recordings with a single frequency pre-identified statistically;
prune the down sampled plurality of ECG recordings to generate a pruned training dataset by eliminating one or more ECG recordings that fail to capture information related to at least one cardiac abnormality among a set of cardiac abnormalities identified as a set scoring classes;
slice each ECG recording in the pruned training data set to generate a windowed dataset of overlapping windows;
apply a fulcrum-based data rebalancing technique on the windowed dataset to generate a rebalanced training dataset for the multiclass-multilabel cardiac abnormalities, the fulcrum-based data rebalancing technique comprising:
deriving a fulcrum point (Nfulcrum) of the windowed dataset from class frequency of each window in the windowed dataset based on one or more class labels from among the set of scoring classes assigned to each window, wherein one or more classes from among the set of scoring classes with class frequency above the fulcrum point are identified as majority classes, wherein classes with class frequency below the fulcrum point are identified as minority classes; and
performing a random sampling on the windowed dataset that is identified under minority classes to reach the fulcrum point, wherein random sampling provides a representation at least equal to the fulcrum point to the minority classes, wherein the majority classes are excluded from the random sampling; and
train a Hybrid Neural Network (HNN) architecture based DL model using the rebalanced training dataset to classify input ECG signals.

### Claim 6 (depends on 5)

The system of claim 5, wherein the one or more hardware processors are configured to utilize the trained DL model to classify an input ECG signal of a subject by predicting the abnormalities in the input ECG into one or more classes among the set of scoring classes.

### Claim 7 (depends on 6)

The system of claim 6, wherein the HNN has three parallel layers comprising:
(i) a Convolutional Neural Network (CNN) layer receiving a multi-lead ECG input of the subject to learn a set of abstracted features, (ii) a Residual Network (ResNet) layer receiving a single lead ECG input to learn discriminating features associated with the heart activity of the subject, and (iii) a layer comprising a set of handcrafted features derived from raw data of the single lead ECG input, and wherein the CNN layer, the ResNet layer and the set of handcrafted features are concatenated to predicting the cardiac abnormalities in the input ECG into one or more classes among the set of scoring classes.

### Claim 8 (depends on 5)

The system of claim 5, wherein the one or more hardware processors are further configured to maintain natural balance of the windowed dataset, control re-sample volume, and support lowly represented classes for training of the HNN via the fulcrum-based data rebalancing technique.

### Claim 9

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:
receiving, a training data set comprising a plurality of ECG recordings with multilabel-multiclass cardiac abnormalities, wherein each of the plurality of ECGs are sampled at varying frequencies;
down sampling, via the one or more hardware processors, the plurality of ECG recordings with a single frequency pre-identified statistically;
pruning, via the one or more hardware processors, the down sampled plurality of ECG recordings to generate a pruned training dataset by eliminating one or more ECG recordings that fail to capture information related to at least one cardiac abnormality among a set of cardiac abnormalities identified as a set scoring classes;
slicing, via the one or more hardware processors, each ECG recording in the pruned training data set to generate a windowed dataset of overlapping windows;
applying, via the one or more hardware processors, a fulcrum-based data rebalancing technique on the windowed dataset to generate a rebalanced training dataset for the multiclass-multilabel cardiac abnormalities, the fulcrum-based data rebalancing technique comprising:
deriving a fulcrum point (Nfulcrum) of the windowed dataset from class frequency of each window in the windowed dataset based on one or more class labels from among the set of scoring classes assigned to each window, wherein one or more classes from among the set of scoring classes with class frequency above the fulcrum point are identified as majority classes, wherein classes with class frequency below the fulcrum point are identified as minority classes; and
performing a random sampling on the windowed dataset that is identified under minority classes to reach the fulcrum point, wherein random sampling provides a representation at least equal to the fulcrum point to the minority classes, wherein the majority classes are excluded from the random sampling; and
training, via the one or more hardware processors, a Hybrid Neural Network (HNN) architecture based DL model using the rebalanced training dataset to classify input ECG signals.

### Claim 10 (depends on 9)

The one or more non-transitory machine-readable information storage mediums of claim 9, wherein the one or more instructions which when executed by the one or more hardware processors further cause utilizing the trained DL model to classify an input ECG signal of a subject by predicting the abnormalities in the input ECG into one or more classes among the set of scoring classes.

### Claim 11 (depends on 10)

The one or more non-transitory machine-readable information storage mediums of claim 10, wherein the one or more instructions which when executed by the one or more hardware processors further cause:
(i) a Convolutional Neural Network (CNN) layer receiving a multi-lead ECG input of the subject to learn a set of abstracted features, (ii) a Residual Network (ResNet) layer receiving a single lead ECG input to learn discriminating features associated with the heart activity of the subject, and (iii) a layer comprising a set of handcrafted features derived from raw data of the single lead ECG input, and wherein the CNN layer, the ResNet layer and the set of handcrafted features are concatenated to predict the cardiac abnormalities in the input ECG into one or more classes among the set of scoring classes.

### Claim 12 (depends on 9)

The one or more non-transitory machine-readable information storage mediums of claim 9, wherein the fulcrum-based data rebalancing technique maintains natural balance of the windowed dataset, controls re-sample volume, and supports lowly represented classes for training of the HNN.
