# US 2022/0015659 A1 · Processing time-frequency representations of eeg data using neural networks

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0015659 A1
- Document ID: US20220015659A1
- Status: Published application
- Kind: A1
- Application number: 16/930,122
- Filed: 2020-07-15
- Published: 2022-01-20
- Earliest claimed: 2020-07-15
- Assignees: X Development LLC
- Applicants: X Development LLC
- Inventors: Mustafa Ispir; Asim Iqbal; Pramod Gupta; Garrett Raymond Honke; Vladimir Miskovic
- Web page: https://patentgrep.com/patent/US20220015659A1

## CPC

- A61B 5/377: using evoked responses
- A61B 5/372: Analysis of electroencephalograms
- A61B 5/726: {using Wavelet transforms}
- A61B 5/7264: {Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N 3/096: Transfer learning
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/30: for calculating health indices; for individual health risk assessment
- G06F 2218/10: by analysing the shape of a waveform, e.g. extracting parameters relating to peaks
- G06F 2218/12: Classification; Matching
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating embeddings of EEG measurements. One of the methods includes obtaining a two-dimensional time-frequency electroencephalogram (EEG) representation corresponding to one or more EEG signal measurements of a user; processing the time-frequency EEG representation using a first neural network having a plurality of first network parameters to generate an embedding of the time-frequency EEG representation, wherein the first neural network has been trained using transfer learning; and providing the embedding of the time-frequency EEG representation to a downstream neural network to generate a mental health prediction for the user.

## Claims

### Claim 1

A method comprising:
obtaining a two-dimensional time-frequency electroencephalogram (EEG) representation corresponding to one or more EEG signal measurements of a user;
processing the time-frequency EEG representation using a first neural network having a plurality of first network parameters to generate an embedding of the time-frequency EEG representation, wherein the first neural network has been trained using transfer learning, the training comprising:
training a second neural network having a plurality of second network parameters to perform an image processing task, comprising updating values for the plurality of second network parameters using a training data set comprising a plurality of training images; and
determining values for the plurality of first network parameters from the trained values of the plurality of second network parameters; and
providing the embedding of the time-frequency EEG representation to a downstream neural network to generate a mental health prediction for the user.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the two-dimensional time-frequency EEG representation is a combined two-dimensional time-frequency EEG representation corresponding to a plurality of EEG signal measurements; and
obtaining the combined two-dimensional time-frequency EEG representation comprises:
obtaining the plurality of EEG signal measurements;
processing each of the plurality of EEG signal measurements to generate respective single-measurement time-frequency EEG representations; and
combining the single-measurement time-frequency EEG representations to generate the combined two-dimensional time-frequency EEG representation.

### Claim 3 (depends on 2)

The method of claim 2, wherein combining the single-measurement time-frequency EEG representations comprises determining a mean or median of the single-measurement time-frequency EEG representations.

### Claim 4 (depends on 2)

The method of claim 2, wherein combining the single-measurement time-frequency EEG representations comprises processing each single-measurement time-frequency EEG representation using a third neural network to generate the combined two-dimensional time-frequency EEG representation.

### Claim 5 (depends on 1)

The method of claim 1, wherein obtaining the two-dimensional time-frequency EEG representation corresponding to one or more EEG signal measurements comprises:
obtaining an EEG signal measurement; and
processing the EEG signal measurement using a plurality of wavelets to generate the two-dimensional time-frequency EEG representation, wherein each of the plurality of wavelets corresponds to a different frequency.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining values for the plurality of first network parameters comprises:
removing a subset of the second network parameters from the second neural network to generate the first neural network, wherein the value for each first network parameter is determined to be equal to a corresponding trained value of the second network parameter.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining values for the plurality of first network parameters comprises:
removing a subset of the second network parameters from the second neural network to generate the first neural network, wherein an initial value for each first network parameter is determined to be equal to a corresponding trained value of the second network parameter; and
updating the initial values of the plurality of first network parameters using an EEG training data set comprising a plurality of time-frequency EEG representations to generate final values for the plurality of first network parameters.

### Claim 8 (depends on 1)

The method of claim 1, wherein the two-dimensional time-frequency EEG representation comprises a plurality of two-dimensional channels each corresponding to a different EEG sensor.

### Claim 9 (depends on 1)

The method of claim 1, wherein each EEG signal measurements corresponds to a same particular prompt of a plurality of prompts of an EEG task.

### Claim 10 (depends on 1)

The method of claim 1, wherein the mental health prediction characterizes a likelihood that the user has a particular mental health disorder.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
providing the embedding of the time-frequency EEG representation to one or more second downstream neural networks to generate respective second mental health predictions for the user, wherein each second mental health prediction characterizes a likelihood that the user has a respective different mental health disorder.

### Claim 12

A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform a method comprising:
obtaining a two-dimensional time-frequency electroencephalogram (EEG) representation corresponding to one or more EEG signal measurements of a user;
processing the time-frequency EEG representation using a first neural network having a plurality of first network parameters to generate an embedding of the time-frequency EEG representation, wherein the first neural network has been trained using transfer learning, the training comprising:
training a second neural network having a plurality of second network parameters to perform an image processing task, comprising updating values for the plurality of second network parameters using a training data set comprising a plurality of training images; and
determining values for the plurality of first network parameters from the trained values of the plurality of second network parameters; and
providing the embedding of the time-frequency EEG representation to a downstream neural network to generate a mental health prediction for the user.

### Claim 13 (depends on 12)

The system of claim 12, wherein:
the two-dimensional time-frequency EEG representation is a combined two-dimensional time-frequency EEG representation corresponding to a plurality of EEG signal measurements; and
obtaining the combined two-dimensional time-frequency EEG representation comprises:
obtaining the plurality of EEG signal measurements;
processing each of the plurality of EEG signal measurements to generate respective single-measurement time-frequency EEG representations; and
combining the single-measurement time-frequency EEG representations to generate the combined two-dimensional time-frequency EEG representation.

### Claim 14 (depends on 12)

The system of claim 12, wherein obtaining the two-dimensional time-frequency EEG representation corresponding to one or more EEG signal measurements comprises:
obtaining an EEG signal measurement; and
processing the EEG signal measurement using a plurality of wavelets to generate the two-dimensional time-frequency EEG representation, wherein each of the plurality of wavelets corresponds to a different frequency.

### Claim 15 (depends on 12)

The system of claim 12, wherein determining values for the plurality of first network parameters comprises:
removing a subset of the second network parameters from the second neural network to generate the first neural network, wherein the value for each first network parameter is determined to be equal to a corresponding trained value of the second network parameter.

### Claim 16 (depends on 12)

The system of claim 12, wherein determining values for the plurality of first network parameters comprises:
removing a subset of the second network parameters from the second neural network to generate the first neural network, wherein an initial value for each first network parameter is determined to be equal to a corresponding trained value of the second network parameter; and
updating the initial values of the plurality of first network parameters using an EEG training data set comprising a plurality of time-frequency EEG representations to generate final values for the plurality of first network parameters.

### Claim 17

One or more non-transitory computer storage media encoded with computer program instructions that when executed by a plurality of computers cause the plurality of computers to perform operations comprising:
obtaining a two-dimensional time-frequency electroencephalogram (EEG) representation corresponding to one or more EEG signal measurements of a user;
processing the time-frequency EEG representation using a first neural network having a plurality of first network parameters to generate an embedding of the time-frequency EEG representation, wherein the first neural network has been trained using transfer learning, the training comprising:
training a second neural network having a plurality of second network parameters to perform an image processing task, comprising updating values for the plurality of second network parameters using a training data set comprising a plurality of training images; and
determining values for the plurality of first network parameters from the trained values of the plurality of second network parameters; and
providing the embedding of the time-frequency EEG representation to a downstream neural network to generate a mental health prediction for the user.

### Claim 18 (depends on 17)

The non-transitory computer storage media of claim 17, wherein:
the two-dimensional time-frequency EEG representation is a combined two-dimensional time-frequency EEG representation corresponding to a plurality of EEG signal measurements; and
obtaining the combined two-dimensional time-frequency EEG representation comprises:
obtaining the plurality of EEG signal measurements;
processing each of the plurality of EEG signal measurements to generate respective single-measurement time-frequency EEG representations; and
combining the single-measurement time-frequency EEG representations to generate the combined two-dimensional time-frequency EEG representation.

### Claim 19 (depends on 17)

The non-transitory computer storage media of claim 17, wherein obtaining the two-dimensional time-frequency EEG representation corresponding to one or more EEG signal measurements comprises:
obtaining an EEG signal measurement; and
processing the EEG signal measurement using a plurality of wavelets to generate the two-dimensional time-frequency EEG representation, wherein each of the plurality of wavelets corresponds to a different frequency.

### Claim 20 (depends on 17)

The non-transitory computer storage media of claim 17, wherein determining values for the plurality of first network parameters comprises:
removing a subset of the second network parameters from the second neural network to generate the first neural network, wherein an initial value for each first network parameter is determined to be equal to a corresponding trained value of the second network parameter; and
updating the initial values of the plurality of first network parameters using an EEG training data set comprising a plurality of time-frequency EEG representations to generate final values for the plurality of first network parameters.
