# US 2024/0252106 A1 · Apparatus and method for determining brain language area invasion based on speech data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0252106 A1
- Document ID: US20240252106A1
- Status: Published application
- Kind: A1
- Application number: 18/406,418
- Filed: 2024-01-08
- Published: 2024-08-01
- Earliest claimed: 2023-01-30
- Assignees: The Catholic University of Korea Industry-Academic Cooperation Foundation; University of Seoul Industry Cooperation Foundation
- Applicants: The Catholic University of Korea Industry-Academic Cooperation Foundation; University of Seoul Industry Cooperation Foundation
- Inventors: Seung Ho YANG; Young Il KIM; Ha Jin YU
- Web page: https://patentgrep.com/patent/US20240252106A1

## CPC

- A61B 5/4803: {Speech analysis specially adapted for diagnostic purposes}
- A61B 5/0042: {for the brain}
- A61B 5/055: involving electronic [EMR] or nuclear [NMR] magnetic resonance, e.g. magnetic resonance imaging
- A61B 5/4064: {Evaluating the brain (for intracranial pressure A61B5/031; for cerebral blood gases A61B5/14553; using EEG A61B5/369)}
- A61B 5/7267: {involving training the classification device}
- G10L 15/063: {Training}
- G10L 15/18: using natural language modelling
- G10L 25/66: for extracting parameters related to health condition (detecting or measuring for diagnostic purposes A61B5/00)
- A61B 2576/026: for the brain
- G10L 13/02: Methods for producing synthetic speech; Speech synthesisers
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Disclosed is a language area invasion determination apparatus including a memory unit including a language area invasion determination model, and a processor that controls an operation of the language area invasion determination model included in the memory unit. The processor trains the language area invasion determination model by using one or more training utterance data and outputs language area invasion determination data of an examiner by using test utterance data and the trained language area invasion determination model. The training utterance data and the test utterance data include utterance speech data of a speaker.

## Claims

### Claim 1

A language area invasion determination apparatus comprising:
a memory unit including a language area invasion determination model; and
a processor configured to control an operation of the language area invasion determination model included in the memory unit,
wherein the processor is configured to:
train the language area invasion determination model by using one or more training utterance data; and
output language area invasion determination data of an examiner by using test utterance data and the trained language area invasion determination model, and
wherein the training utterance data and the test utterance data include utterance speech data of a speaker.

### Claim 2 (depends on 1)

The language area invasion determination apparatus of claim 1, wherein the training utterance data further includes the language area invasion determination data, and
wherein the language area invasion determination data includes at least one of language area invasion-related data or detailed invasion area data.

### Claim 3 (depends on 2)

The language area invasion determination apparatus of claim 2, wherein the language area invasion-related data or the detailed invasion area data includes brain image-based data of the speaker.

### Claim 4 (depends on 3)

The language area invasion determination apparatus of claim 3, wherein the brain image-based data includes a brain image of the speaker or brain image read data, and
wherein the brain image includes at least one of functional MRI (fMRI), DTI tractography, and intraoperative mapping.

### Claim 5 (depends on 1)

The language area invasion determination apparatus of claim 1, wherein the training utterance data includes original utterance data, modulation utterance data, and utterance data generated by using an artificial intelligence-based text-to-speech.

### Claim 6 (depends on 1)

The language area invasion determination apparatus of claim 1, wherein the language area invasion determination model is built by using one or more neural networks, and applies an ensemble technique to the one or more neural networks.

### Claim 7 (depends on 6)

The language area invasion determination apparatus of claim 6, wherein the applying of the ensemble technique to the one or more neural networks includes:
inputting, by the language area invasion determination model, utterance data into the one or more neural networks, and ensembling language area invasion determination data generated from each of the one or more neural networks, and
wherein the one or more neural networks use a recurrent neural network, a convolutional neural network, a transformer, and a long short-term memory.

### Claim 8 (depends on 7)

The language area invasion determination apparatus of claim 7, wherein the utterance data further includes utterance transcript data.

### Claim 9 (depends on 8)

The language area invasion determination apparatus of claim 8, wherein the language area invasion determination model further includes a sentence reading network that receives the utterance transcript data and generates language area invasion determination data.

### Claim 10 (depends on 9)

The language area invasion determination apparatus of claim 9, wherein the sentence reading network outputs detailed invasion area data.

### Claim 11 (depends on 1)

The language area invasion determination apparatus of claim 1, wherein the processor is configured to:
train an utterance function evaluation model of the speaker by using the training utterance data, and
wherein the utterance function evaluation model calculates a similarity between normal utterance data and the training utterance data or the test utterance data.

### Claim 12 (depends on 11)

The language area invasion determination apparatus of claim 11, wherein the calculating of the similarity between the normal utterance data and the training utterance data or the test utterance data includes:
calculating a similarity between a waveform of the normal utterance data and a waveform of the training utterance data or a waveform of the test utterance data.

### Claim 13

A method performed by a language area invasion determination apparatus, method comprising:
training a language area invasion determination model by using one or more training utterance data; and
outputting language area invasion determination data of an examiner by using test utterance data and the trained language area invasion determination model, and
wherein the training utterance data and the test utterance data include utterance speech data of a speaker.

### Claim 14 (depends on 13)

The method of claim 13, wherein the training utterance data further includes the language area invasion determination data, and
wherein the language area invasion determination data includes at least one of language area invasion-related data or detailed invasion area data.

### Claim 15 (depends on 14)

The method of claim 14, wherein the language area invasion-related data or the detailed invasion area data includes brain image-based data of the speaker.

### Claim 16 (depends on 15)

The method of claim 15, wherein the brain image-based data includes a brain image of the speaker or brain image read data, and
wherein the brain image includes at least one of fMRI, DTI tractography, and intraoperative mapping.

### Claim 17 (depends on 13)

The method of claim 13, wherein the training utterance data includes original utterance data, modulation utterance data, and utterance data generated by using an artificial intelligence-based text-to-speech.

### Claim 18 (depends on 13)

The method of claim 13, wherein the language area invasion determination model is built by using one or more neural networks, and applies an ensemble technique to the one or more neural networks.

### Claim 19 (depends on 18)

The method of claim 18, wherein the applying of the ensemble technique to the one or more neural networks includes:
inputting, by the language area invasion determination model, utterance data into the one or more neural networks, and ensembling language area invasion determination data generated from each of the one or more neural networks, and
wherein the one or more neural networks use a recurrent neural network, a convolutional neural network, a transformer, and a long short-term memory.

### Claim 20 (depends on 19)

The method of claim 19, wherein the utterance data further includes utterance transcript data,
wherein the language area invasion determination model further includes a sentence reading network that receives the utterance transcript data and generates language area invasion determination data, and
wherein the sentence reading network outputs detailed invasion area data.
