# US 2021/0280169 A1 · Metric learning of speaker diarization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0280169 A1
- Document ID: US20210280169A1
- Status: Published application
- Kind: A1
- Application number: 16/807,892
- Filed: 2020-03-03
- Published: 2021-09-09
- Earliest claimed: 2020-03-03
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Masayuki Suzuki; Yosuke Higuchi
- Web page: https://patentgrep.com/patent/US20210280169A1

## CPC

- G10L 15/063: {Training}
- G10L 17/04: Training, enrolment or model building
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 25/51: for comparison or discrimination
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A computer-implemented method includes obtaining training data including utterances of speakers in acoustic conditions, preparing at least one machine learning model, each machine learning model including a common embedding model for converting an utterance into a feature vector and a classification model for classifying the feature vector, and training, by using the training data, the machine learning model to perform classification by speaker and to perform classification by acoustic condition.

## Claims

### Claim 1

A computer-implemented method comprising:
obtaining training data including a plurality of utterances of a plurality of speakers in a plurality of acoustic conditions;
preparing at least one machine learning model, each machine learning model including a common embedding model for converting an utterance into a feature vector and a classification model for classifying the feature vector; and
training, by using the training data, the at least one machine learning model to perform classification by speaker and to perform classification by acoustic condition.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the training includes training the at least one machine learning model until the embedding model is trained to produce a conversion function for relating utterances to feature vectors including characteristics of speakers and acoustic conditions.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, further comprising producing a converter that converts an utterance into a feature vector by implementing the trained embedding model as a conversion function of the converter.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the training includes
performing, by using the training data, a first task to train a first machine learning model including the embedding model and a first classification model for classifying the feature vector by speaker; and
performing, by using the training data, a second task to train a second machine learning model including the embedding model and a second classification model for classifying the feature vector by acoustic condition.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein
the performing the first task is for training the first machine learning model without using information of acoustic condition in the training data; and
the performing the second task is for training the second machine learning model without using information of speaker in the training data.

### Claim 6 (depends on 4)

The computer-implemented method of claim 4, wherein the first task and the second task are performed according to a multi-task training technique.

### Claim 7 (depends on 4)

The computer-implemented method of claim 4, wherein the obtaining the training data includes assigning a common identifier of an acoustic condition to each utterances of a common speaker obtained from a single continuous recording.

### Claim 8 (depends on 5)

The computer-implemented method of claim 5, wherein the training includes performing, by using the training data, a third task to train a third machine learning model including the embedding model and a third classification model for classifying the feature vector by speaker and acoustic condition.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the obtaining the training data further includes generating, for each utterance, a training output data of the third classification model by extracting common features of utterances among the plurality of utterances having a common speaker and a common acoustic condition, and
wherein the performing the third task includes training the third machine learning model to output the training output data representing a speaker and an acoustic condition from an utterance of a speaker and an acoustic condition.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein the generating includes generating an i-vector as the training output data.

### Claim 11

A computer program product including one or more computer-readable storage mediums collectively storing program instructions that are executable by a processor or programmable circuitry to cause the processor or programmable circuitry to perform operations comprising:
obtaining training data including a plurality of utterances of a plurality of speakers in a plurality of acoustic conditions;
preparing at least one machine learning model, each machine learning model including a common embedding model for converting an utterance into a feature vector and a classification model for classifying the feature vector; and
training, by using the training data, the at least one machine learning model to perform classification by speaker and to perform classification by acoustic condition.

### Claim 12 (depends on 11)

The computer program product of claim 11, wherein the operations further comprise producing a converter that converts an utterance into a feature vector by implementing the trained embedding model as a conversion function of the converter.

### Claim 13 (depends on 11)

The computer program product of claim 11, wherein the training includes
performing, by using the training data, a first task to train a first machine learning model including the embedding model and a first classification model for classifying the feature vector by speaker; and
performing, by using the training data, a second task to train a second machine learning model including the embedding model and a second classification model for classifying the feature vector by acoustic condition.

### Claim 14 (depends on 13)

The computer program product of claim 13, wherein the training includes performing, by using the training data, a third task to train a third machine learning model including the embedding model and a third classification model for classifying the feature vector by speaker and acoustic condition.

### Claim 15 (depends on 14)

The computer program product of claim 14, wherein the obtaining the training data further includes generating, for each utterance, a training output data of the third classification model by extracting common features of utterances among the plurality of utterances having a common speaker and a common acoustic condition, and
wherein the performing the third task includes training the third machine learning model to output the training output data representing a speaker and an acoustic condition from an utterance of a speaker and an acoustic condition.

### Claim 16

An apparatus comprising:
a processor or a programmable circuitry; and
one or more computer-readable mediums collectively including instructions that, when executed by the processor or the programmable circuitry, cause the processor or the programmable circuitry to:
obtain training data including a plurality of utterances of a plurality of speakers in a plurality of acoustic conditions;
prepare at least one machine learning model, each machine learning model including a common embedding model for converting an utterance into a feature vector and a classification model for classifying the feature vector; and
train by using the training data, the at least one machine learning model to perform classification by speaker and to perform classification by acoustic condition.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein executing the instructions further causes the processor or the programmable circuitry to produce a converter that converts an utterance into a feature vector by implementing the trained embedding model as a conversion function of the converter.

### Claim 18 (depends on 16)

The apparatus of claim 16, wherein the training includes
performing, by using the training data, a first task to train a first machine learning model including the embedding model and a first classification model for classifying the feature vector by speaker; and
performing, by using the training data, a second task to train a second machine learning model including the embedding model and a second classification model for classifying the feature vector by acoustic condition.

### Claim 19 (depends on 18)

The apparatus of claim 18, wherein the training includes performing, by using the training data, a third task to train a third machine learning model including the embedding model and a third classification model for classifying the feature vector by speaker and acoustic condition.

### Claim 20 (depends on 19)

The apparatus of claim 19, wherein the obtaining the training data further includes generating, for each utterance, a training output data of the third classification model by extracting common features of utterances among the plurality of utterances having a common speaker and a common acoustic condition, and
wherein the performing the third task includes training the third machine learning model to output the training output data representing a speaker and an acoustic condition from an utterance of a speaker and an acoustic condition.
