# US 11,443,748 B2 · Metric learning of speaker diarization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,748 B2 - Document ID: US11443748B2 - Status: Granted - Kind: B2 - Application number: 16/807,818 - Filed: 2020-03-03 - Published: 2021-09-09 - Granted: 2022-09-13 - Earliest claimed: 2020-03-03 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Masayuki Suzuki - Web page: https://patentgrep.com/patent/US11443748B2 ## CPC - G10L 17/04: Training, enrolment or model building - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06N 3/0985: Hyperparameter optimisation; Meta-learning; Learning-to-learn - G06N 20/00: Machine learning - G10L 17/00: Speaker identification or verification techniques - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A computer-implemented method includes obtaining, using a hardware processor, training data including utterances of speakers and performing tasks to train a machine learning model that converts an utterance into a feature vector, each task using one subset of multiple subsets of training data. The subsets of training data include a first subset of training data including utterances of a first number of speakers and at least one second subset of training data. Each second subset of training data includes utterances of a number of speakers that is less than the first number of speakers. ## Claims ### Claim 1 A computer-implemented method comprising: obtaining, using a hardware processor, training data stored on one or more computer readable storage mediums, the training data including a plurality of utterances of a plurality of speakers; and performing, using the hardware processor, a plurality of tasks to train a machine learning model that converts an utterance of the plurality of utterances into a feature vector, each task using one of a plurality of subsets of training data, wherein the plurality of subsets of training data includes: a first subset of training data including utterances of a first number of speakers among the plurality of speakers, and at least one second subset of training data, each second subset including utterances of a number of speakers among the plurality of speakers that is less than the first number of speakers among the plurality of speakers, wherein performing the plurality of tasks to train the machine learning model includes performing one task using the first subset of training data and two or more tasks using the at least one second subset of training data. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein performing the plurality of tasks to train the machine learning model includes performing the plurality of tasks according to a multi-task training technique. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the machine learning model includes a first model for converting an utterance of the plurality of utterances into a feature vector and a second model for identifying a speaker of the plurality of speakers from a feature vector, and each utterance of the plurality of utterances in the training data is paired with an identification of a speaker of the plurality of speakers corresponding thereto. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, further comprising producing, using the hardware processor, a converter that converts an utterance of a speaker of the plurality of speakers into a feature vector by training the first model. ### Claim 5 (depends on 3) The computer-implemented method of claim 3, wherein performing the plurality of tasks includes using a value of at least one hyperparameter of the task using the first subset of training data that is different from the value of the at least one hyperparameter of the task using the at least one second subset of training data. ### Claim 6 (depends on 5) The computer-implemented method of claim 5, wherein the at least one hyperparameter is a margin of loss function. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the utterances of the second subset of training data are recorded in a substantially similar acoustic environment. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein the utterances of the second subset of training data are obtained from a single continuous recording. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the utterances of the first subset of training data are obtained by combining two or more audio recordings. ### Claim 10 A computer program product including one or more computer-readable storage mediums collectively storing program instructions that are executable by a processor or programmable circuitry to cause the processor or programmable circuitry to perform operations comprising: obtaining training data including a plurality of utterances of a plurality of speakers; and performing a plurality of tasks to train a machine learning model that converts an utterance of the plurality of utterances into a feature vector, each task using one of a plurality of subsets of training data, wherein the plurality of subsets of training data includes: a first subset of training data including utterances of a first number of speakers among the plurality of speakers, and at least one second subset of training data, each second subset including utterances of a number of speakers among the plurality of speakers that is less than the first number of speakers among the plurality of speakers, wherein performing the plurality of tasks to train the machine learning model includes performing one task using the first subset of training data and two or more tasks using the at least one second subset of training data. ### Claim 11 (depends on 10) The computer program product of claim 10, wherein the machine learning model includes a first model for converting an utterance of the plurality of utterances into a feature vector and a second model for identifying a speaker of the plurality of speakers from a feature vector, and each utterance of the plurality of utterances in the training data is paired with an identification of a speaker of the plurality of speakers corresponding thereto. ### Claim 12 (depends on 11) The computer program product of claim 11, wherein the operations further comprise producing a converter that converts an utterance of a speaker of the plurality of speakers into a feature vector by training the first model. ### Claim 13 (depends on 11) The computer program product of claim 11, wherein performing the plurality of tasks includes using a value of at least one hyperparameter of the task using the first subset of training data that is different from the value of the at least one hyperparameter of the task using the at least one second subset of training data. ### Claim 14 (depends on 10) The computer program product of claim 10, wherein the utterances of the second subset of training data are obtained from a single continuous recording. ### Claim 15 An apparatus comprising: a processor or programmable circuitry; and one or more computer readable storage mediums collectively including instructions that, when executed by the processor or the programmable circuitry, cause the processor or the programmable circuitry to: obtain training data including a plurality of utterances of a plurality of speakers; and perform a plurality of tasks to train a machine learning model that converts an utterance of the plurality of utterances into a feature vector, each task using one of a plurality of subsets of training data, wherein the plurality of subsets of training data includes: a first subset of training data including utterances of a first number of speakers among the plurality of speakers, and at least one second subset of training data, each second subset including utterances of a number of speakers among the plurality of speakers that is less than the first number of speakers among the plurality of speakers, wherein performing the plurality of tasks to train the machine learning model includes performing one task using the first subset of training data and two or more tasks using the at least one second subset of training data. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein the machine learning model includes a first model for converting an utterance of the plurality of utterances into a feature vector and a second model for identifying a speaker of the plurality of speakers from a feature vector, and each utterance of the plurality of utterances in the training data is paired with an identification of a speaker of the plurality of speakers corresponding thereto. ### Claim 17 (depends on 16) The apparatus of claim 16, wherein the instructions further cause the processor or the programmable circuitry to produce a converter that converts an utterance of a speaker of the plurality of speakers into a feature vector by training the first model. ### Claim 18 (depends on 16) The apparatus of claim 16, wherein performing the plurality of tasks includes using a value of at least one hyperparameter of the task using the first subset of training data that is different from the value of the at least one hyperparameter of the task using the at least one second subset of training data. ### Claim 19 (depends on 15) The apparatus of claim 15, wherein the utterances of the second subset of training data are obtained from a single continuous recording.