# US 2021/0241908 A1 · Multi-sensor based hmi/ai-based system for diagnosis and therapeutic treatment of patients with neurological disease

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0241908 A1
- Document ID: US20210241908A1
- Status: Published application
- Kind: A1
- Application number: 17/050,702
- Filed: 2019-04-24
- Published: 2021-08-05
- Earliest claimed: 2018-04-26
- Applicants: Mindmaze Holding SA
- Inventors: Martin CIUPA; Tej TADI
- Web page: https://patentgrep.com/patent/US20210241908A1

## CPC

- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- A61B 5/1116: {Determining posture transitions}
- A61B 5/1176: {Recognition of faces}
- A61B 5/369: Electroencephalography [EEG]
- A61B 5/389: Electromyography [EMG]
- A61B 5/4082: {Diagnosing or monitoring movement diseases, e.g. Parkinson, Huntington or Tourette}
- A61B 5/4088: {Diagnosing of monitoring cognitive diseases, e.g. Alzheimer, prion diseases or dementia}
- A61B 5/4094: {Diagnosing or monitoring seizure diseases, e.g. epilepsy}
- A61B 5/4803: {Speech analysis specially adapted for diagnostic purposes}
- A61B 5/7203: {for noise prevention, reduction or removal}
- A61B 5/7257: {using Fourier transforms}
- A61B 5/726: {using Wavelet transforms}
- A61B 5/7264: {Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems}
- A61B 5/7275: {Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor}
- A61B 6/032: {Transmission computed tomography [CT]}
- A61B 6/037: {Emission tomography}
- G06F 17/142: {Fast Fourier transforms, e.g. using a Cooley-Tukey type algorithm}
- G06F 17/147: {Discrete orthonormal transforms, e.g. discrete cosine transform, discrete sine transform, and variations therefrom, e.g. modified discrete cosine transform, integer transforms approximating the discrete cosine transform (G06F17/145 takes precedence)}
- G06N 20/00: Machine learning
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G16H 20/00: ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A system and method for diagnosis and therapeutic treatment of patients with neurological disease from a plurality of sensors, and in particular, to such a system, method and apparatus for analyzing data from a plurality of sensors with an AI (artificial intelligence) algorithm.

## Claims

### Claim 1

A system for determining a prognosis of a neurological disease in a subject, the system comprising a plurality of sensors for obtaining measurements of the subject, a computational device comprising a processor and memory, wherein instructions are stored on said memory for execution by said processor, wherein said computational device receives sensor data from said sensors and analyzes said sensor data to determine one or more states of the subject, wherein said sensor data is analyzed by a machine learning or deep learning engine, wherein said engine comprises one or more machine learning or deep learning algorithms trained to analyze said sensor data and to determine a prognosis of the neurological disease of the subject from said sensor data, wherein said algorithms are executed by said processor according to instructions stored on said memory.

### Claim 2 (depends on 1)

The system of claim 1, wherein said processor is configured to perform a defined set of basic operations in response to receiving a corresponding basic instruction selected from a defined native instruction set of codes; said computational device comprising a first set of machine codes stored in the memory and selected from the native instruction set for receiving said sensor data, a second set of machine codes stored in the memory and selected from the native instruction set for processing said sensor data to form processed sensor data; and a third set of machine codes stored in the memory and selected from the native instruction set for executing said one or more machine learning or deep learning algorithms to analyze said processed sensor data.

### Claim 3 (depends on 1)

The system of claim 1, wherein said memory stores instructions for enabling said processor to process said sensor data to form processed sensor data, wherein said instructions include transforming said sensor data according to one or more of FFT (Fast Fourier Transform), short FFT, DCT (discrete cosine transform), fast discrete cosine transform, DFT (discrete Fourier Transform), DWT (discrete wavelet transform) or Hilbert space transforms.

### Claim 4 (depends on 3)

The system of claim 3, wherein said instructions further comprise instructions for transforming a plurality of different types of sensor data to a consistent Hilbert space to form common Hilbert space data.

### Claim 5 (depends on 4)

The system of claim 4, wherein said instructions for executing said one or more machine learning or deep learning algorithms to analyze said processed sensor data further comprise instructions for receiving a plurality of different types of sensor data as said common Hilbert space data and analyzing said common Hilbert space data according to said one or more machine learning or deep learning algorithms.

### Claim 6 (depends on 5)

The system of claim 5, wherein said machine learning algorithms comprise one or more of Naive Bayesian algorithm, Bagging classifier, SVM (support vector machine) classifier, NC (node classifier), NCS (neural classifier system), SCRLDA (Shrunken Centroid Regularized Linear Discriminate and Analysis), Random Forest.

### Claim 7 (depends on 6)

The system of claim 6, wherein said deep learning algorithms comprise one or more of a CNN (convolutional neural network), RNN (recurrent neural network), DBN (deep belief network), and GAN (generalized adversarial network).

### Claim 8 (depends on 7)

The system of claim 7, wherein said instructions further comprise instructions for denoising said sensor data before analyzing said sensor data.

### Claim 9 (depends on 7)

The system of claim 7, wherein said instructions further comprise instructions for receiving medical record data and transforming said medical record data to structured data.

### Claim 10 (depends on 9)

The system of claim 9, wherein said instructions further comprise instructions for analyzing said structured data by said one or more machine learning or deep learning algorithms.

### Claim 11 (depends on 7)

The system of claim 7, wherein said instructions further comprise instructions for determining a diagnosis for the subject by determining a location of the subject on a diagnostic tree according to said analyzed data.

### Claim 12 (depends on 11)

The system of claim 11, wherein said instructions further comprise instructions for converting said data to a unified data structure, mapping said unified data structure information to data characteristics and applying a machine learning model, to determine said location on said diagnostic tree.

### Claim 13 (depends on 12)

The system of claim 12, wherein said instructions further comprise instructions for comparing said location on said diagnostic tree for said subject to locations on said diagnostic tree for additional subjects, wherein said prognosis for said additional subjects is known, to determine said prognosis for the subject.

### Claim 14 (depends on 7)

The system of claim 7, wherein said instructions further comprise instructions for determining a treatment path for the subject according to a diagnosis and said prognosis.

### Claim 15 (depends on 7)

The system of claim 7, wherein the neurological disease is selected from the group consisting of stroke, dementia, epilepsy, multiple sclerosis, neuroinfection, ALS, Parkinson's disease and traumatic brain injuries.

### Claim 16 (depends on 7)

The system of claim 7, further comprising adjusting said treatment of the subject according to a plurality of additional sensor measurements.

### Claim 17 (depends on 7)

The system of claim 7, wherein said prognosis of the subject is determined according to one or more of facial expression or micro-expression; EEG (electroencephalography) activity; voice related features; determining the position and/or movement of the subject.

### Claim 18 (depends on 7)

The system of claim 7, wherein said sensor data relates to one or more of video data, audio data, EMG (electromyography) data, EEG data, ECG, TOF (time of flight) data, other optical data, depth data, and imaging modality data.

### Claim 19 (depends on 18)

The system of claim 18, wherein said imaging modality data comprises one or more of PET, MRI, fMRI, SPECT and CAT data.

### Claim 20 (depends on 7)

The system of claim 7, wherein said computational device further comprises a data translation layer for translating said data for ingestion by said engine.

### Claim 21 (depends on 20)

The system of claim 20, wherein said data translation layer combines a plurality of different data types from different sensors into a consistent multi-dimensional vector space.

### Claim 22 (depends on 21)

The system of claim 21, wherein said data translation layer denoises said sensor data before combining said plurality of different data sensor types.

### Claim 23 (depends on 22)

The system of claim 22, wherein said data in said consistent multi-dimensional vector space is fed into a single machine learning or deep learning algorithm.

### Claim 24 (depends on 23)

The system of claim 23, wherein said data in said consistent multi-dimensional vector space is fed into a plurality of machine learning or deep learning algorithms.

### Claim 25 (depends on 24)

The system of claim 24, wherein results from said plurality of machine learning or deep learning algorithms are combined through a single machine learning or deep learning algorithm to determine said prognosis.

### Claim 26 (depends on 7)

The system of claim 7, wherein said machine learning or deep learning algorithm comprises a CNN (convolutional neural network) receiving a plurality of data types in a plurality of convolutional and pooling layers, featuring a single connected layer for determining said prognosis of the subject.