# US 2021/0280197 A1 · Fully Supervised Speaker Diarization > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0280197 A1 - Document ID: US20210280197A1 - Status: Published application - Kind: A1 - Application number: 17/303,283 - Filed: 2021-05-26 - Published: 2021-09-09 - Earliest claimed: 2019-01-08 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Chong Wang; Aonan Zhang; Quan Wang; Zhenyao Zhu - Web page: https://patentgrep.com/patent/US20210280197A1 ## CPC - G10L 17/04: Training, enrolment or model building - G10L 15/04: Segmentation; Word boundary detection - G10L 15/075: {supervised, i.e. under machine guidance} - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G10L 17/00: Speaker identification or verification techniques - G10L 17/02: Preprocessing operations, e.g. segment selection; Pattern representation or modelling, e.g. based on linear discriminant analysis [LDA] or principal components; Feature selection or extraction - G10L 17/18: Artificial neural networks; Connectionist approaches - G10L 25/30: using neural networks - G10L 25/87: Detection of discrete points within a voice signal - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract A method includes receiving an utterance of speech and segmenting the utterance of speech into a plurality of segments. For each segment of the utterance of speech, the method also includes extracting a speaker=discriminative embedding from the segment and predicting a probability distribution over possible speakers for the segment using a probabilistic generative model configured to receive the extracted speaker-discriminative embedding as a feature input. The probabilistic generative model trained on a corpus of training speech utterances each segmented into a plurality of training segments. Each training segment including a corresponding speaker-discriminative embedding and a corresponding speaker label. The method also includes assigning a speaker label to each segment of the utterance of speech based on the probability distribution over possible speakers for the corresponding segment. ## Claims ### Claim 1 A computer-implemented method when executed on data processing hardware causes the data processing hardware to perform operations comprising: receiving audio data corresponding to an utterance of speech; segmenting the audio data into a plurality of segments; and for a current speech segment occurring after an initial speech segment of the plurality of segments: extracting a current speaker-discriminative embedding from the current speech segment; predicting, using a neural network model configured to receive the current speaker-discriminative embedding extracted from the current speech segment as input, a probability distribution over possible speakers for the current speech segment by: for each of one or more existing speakers associated with a corresponding speaker label, predicting a probability that the current speaker-discriminative embedding includes the existing speaker based on a corresponding existing speaker-discriminative embedding that is assigned to the corresponding speaker label associated with the existing speaker; and predicting a probability that the current speaker-discriminative embedding includes a new speaker; and based on the probability distribution over possible speakers for the current speech segment, assigning, to the current speaker-discriminative embedding, a current speaker label as one of: a new speaker label associated with the new speaker; or the corresponding speaker label associated with one of the one or more existing speakers. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein segmenting the audio data comprises segmenting the audio data into a plurality of non-overlapping speech segments. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein segmenting the audio data comprises segmenting the audio data using a voice activity detector (VAD) configured to segment the audio data into the plurality of non-overlapping speech segments. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, wherein the operations further comprise, for the initial speech segment of the plurality of speech segments: extracting a first speaker-discriminative embedding from the initial speech segment; and assigning a first speaker label associated with a first speaker to the first speaker-discriminative embedding. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein extracting the current speaker-discriminative embedding from the current speech segment comprises extracting a d-vector from the current speech segment. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein extracting the current speaker-discriminative embedding from the current speech segment comprises extracting an i-vector from the current speech segment. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the operations further comprise: transcribing the utterance of speech into corresponding text; and annotating the text based on the current speaker label. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, wherein segmenting the audio data into a plurality of segments comprises segmenting the audio data into a plurality of fixed-length segments. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein segmenting the audio data into a plurality of segments comprises segmenting the audio data into a plurality of variable-length segments. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein the data processing hardware resides on a user device or a distributed system. ### Claim 11 A system comprising: data processing hardware; memory hardware in communication with the data processing hardware and storing instructions, that when executed by the data processing hardware, cause the data processing hardware to perform operations comprising: receiving audio data corresponding to an utterance of speech; segmenting the audio data into a plurality of segments; and for a current speech segment occurring after an initial speech segment of the plurality of segments: extracting a current speaker-discriminative embedding from the current speech segment; predicting, using a neural network model configured to receive the current speaker-discriminative embedding extracted from the current speech segment as input, a probability distribution over possible speakers for the current speech segment by: for each of one or more existing speakers associated with a corresponding speaker label, predicting a probability that the current speaker-discriminative embedding includes the existing speaker based on a corresponding existing speaker-discriminative embedding that is assigned to the corresponding speaker label associated with the existing speaker; and predicting a probability that the current speaker-discriminative embedding includes a new speaker; and based on the probability distribution over possible speakers for the current speech segment, assigning, to the current speaker-discriminative embedding, a current speaker label as one of: a new speaker label associated with the new speaker; or the corresponding speaker label associated with one of the one or more existing speakers. ### Claim 12 (depends on 11) The system of claim 11, wherein segmenting the audio data comprises segmenting the audio data into a plurality of non-overlapping speech segments. ### Claim 13 (depends on 12) The system of claim 12, wherein segmenting the audio data comprises segmenting the audio data using a voice activity detector (VAD) configured to segment the audio data into the plurality of non-overlapping speech segments. ### Claim 14 (depends on 11) The system of claim 11, wherein the operations further comprise, for the initial speech segment of the plurality of speech segments: extracting a first speaker-discriminative embedding from the initial speech segment; and assigning a first speaker label associated with a first speaker to the first speaker-discriminative embedding. ### Claim 15 (depends on 11) The system of claim 11, wherein extracting the current speaker-discriminative embedding from the current speech segment comprises extracting a d-vector from the current speech segment. ### Claim 16 (depends on 11) The system of claim 11, wherein extracting the current speaker-discriminative embedding from the current speech segment comprises extracting an i-vector from the current speech segment. ### Claim 17 (depends on 11) The system of claim 11, wherein the operations further comprise: transcribing the utterance of speech into corresponding text; and annotating the text based on the current speaker label. ### Claim 18 (depends on 11) The system of claim 11, wherein segmenting the audio data into a plurality of segments comprises segmenting the audio data into a plurality of fixed-length segments. ### Claim 19 (depends on 11) The system of claim 11, wherein segmenting the audio data into a plurality of segments comprises segmenting the audio data into a plurality of variable-length segments. ### Claim 20 (depends on 11) The system of claim 11, wherein the data processing hardware resides on a user device or a distributed system.