# US 2024/0203407 A1 · Automatic generation of a contextual meeting summary

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0203407 A1
- Document ID: US20240203407A1
- Status: Published application
- Kind: A1
- Application number: 18/590,695
- Filed: 2024-02-28
- Published: 2024-06-20
- Earliest claimed: 2021-11-02
- Assignees: Capital One Services, LLC
- Applicants: Capital One Services, LLC
- Inventors: Grant EDEN; Jeremy GOODSITT; Austin WALTERS; Anh TRUONG
- Web page: https://patentgrep.com/patent/US20240203407A1

## CPC

- G10L 15/183: using context dependencies, e.g. language models
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G06F 40/35: Discourse or dialogue representation
- G06N 20/00: Machine learning
- G10L 17/06: Decision making techniques; Pattern matching strategies
- G10L 17/26: Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
- G10L 25/63: for estimating an emotional state
- G10L 25/30: using neural networks
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods are disclosed herein for building contextual transcripts. A computing system may receive a textual transcript of a meeting that contains a variety of statements made by various attendees of the meeting, select the first statement made during the meeting, and determine which meeting attendee made the statement. A machine learning model corresponding to the particular attendee that has been trained using previously received statements by the particular attendee may be used on the utterance to determine the tone of the utterance. That tone may be recorded within the transcript and this process may be repeated for each utterance to build a contextual transcript.

## Claims

### Claim 1

A system for generating a identifying speakers, the system comprising:
one or more processors; and
a non-transitory computer-readable storage medium storing instructions, which when executed by the one or more processors cause the one or more processors to:
receiving a voice recording of a meeting, wherein the voice recording comprises a plurality of utterances for a plurality of participants attending the meeting and a corresponding time range for each utterance of the plurality of utterances;
retrieving a list of participants for the meeting;
for each utterance of the plurality of utterances, extracting, using the corresponding time range, corresponding audio associated with each utterance;
comparing each utterance with voice prints of participants in the list of participants;
matching, based on comparing each utterance with the voice prints of the participants in the list of participants, each utterance with a corresponding participant; and
updating a data structure associated with the meeting to include an identifier of the corresponding participant.

### Claim 2 (depends on 1)

The system of claim 1, wherein the instructions further cause the one or more processors to store the identifier of the corresponding participant, a corresponding time period and a textual representation of a corresponding utterance.

### Claim 3 (depends on 1)

The system of claim 1, wherein the instructions further cause the one or more processors to:
determine, using a directional machine learning model, that a first utterance of the plurality of utterances is a response to a previous utterance;
identify a speaker of the previous utterance; and
modify the data structure to include the speaker of the previous utterance as a direction of the first utterance.

### Claim 4 (depends on 1)

The system of claim 1, wherein the instructions cause the one or more processors to:
transmit a first command to a device associated with a first user to store a first tonal machine learning model on the device associated with the first user;
transmit a second command to the device associated with the first user to continuously train the first tonal machine learning model as the first user interacts with the device associated with the first user; and
periodically, cause the first tonal machine learning model to be transmitted from the device associated with the first user.

### Claim 5 (depends on 1)

The system of claim 1, wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:
select a tonal machine learning model associated with a user corresponding to an utterance by:
retrieving a first device address associated with a first device of a first user;
transmitting, to the first device address, a command to transmit a first tonal machine learning model, wherein the first tonal machine learning model is associated with the first user; and
receiving the first tonal machine learning model from the first device.

### Claim 6

A method comprising:
receiving a voice recording of a meeting, wherein the voice recording comprises a plurality of utterances for a plurality of participants attending the meeting and a corresponding time range for each utterance of the plurality of utterances;
retrieving a list of participants for the meeting;
for each utterance of the plurality of utterances, extracting, using the corresponding time range, corresponding audio associated with each utterance;
comparing each utterance with voice prints of participants in the list of participants;
matching, based on comparing each utterance with the voice prints of the participants in the list of participants, each utterance with a corresponding participant; and
generating a data structure associated with the meeting to include an identifier of the corresponding participant.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
generating, using a speech recognition algorithm, a textual transcript of the voice recording, wherein the textual transcript comprises corresponding text of each utterance and the corresponding time range within the voice recording for each utterance; and
adding, to the data structure, a plurality of fields storing for each utterance the corresponding text and the corresponding time range.

### Claim 8 (depends on 6)

The method of claim 6, further comprising:
generating, from a portion of a recording associated with a first utterance of the plurality of utterances, a plurality of voice characteristics for the first utterance, the plurality of voice characteristics comprising an amplitude and a frequency;
generating an input vector for the first utterance, the input vector comprising the amplitude as the frequency; and
assigning the input vector to utterance data associated with the first utterance.

### Claim 9 (depends on 6)

The method of claim 6, further comprising:
converting a textual representation of a first utterance of the plurality of utterances into a first vector representation;
converting an audio data associated with the first utterance into a second vector representation; and
assigning the first vector representation and the second vector representation to utterance data associated with the first utterance.

### Claim 10 (depends on 6)

The method of claim 6, further comprising:
receiving a textual transcript of the meeting, wherein the textual transcript comprises the plurality of utterances for a plurality of users attending the meeting;
selecting a first utterance of the plurality of utterances;
determining, for the first utterance of the plurality of utterances, a user of the plurality of users associated with the first utterance;
selecting, from a plurality of machine learning models associated with the plurality of users, a first machine learning model associated with the user, wherein each machine learning model was trained, using a training set of utterances of a corresponding user, to determine a corresponding tone associated with a particular utterance;
inputting utterance data associated with the first utterance into the first machine learning model to obtain a tone associated with the first utterance, wherein the utterance data comprises voice characteristics associated with the first utterance and textual data of the first utterance; and
modifying a contextual transcript comprising an entry for the first utterance with the tone associated with the first utterance.

### Claim 11 (depends on 10)

The method of claim 10, wherein selecting the first machine learning model associated with the user comprises:
retrieving a first device address associated with a first device of a first user;
transmitting, to the first device address, a command to the first device of the first user, wherein the command instructs the first device of the user to transmit a first tonal machine learning model, wherein the first tonal machine learning model is associated with the first user; and
receiving the first tonal machine learning model from the first device.

### Claim 12 (depends on 6)

The method of claim 6, further comprising:
storing a first machine learning model on a first device associated with a first user, wherein the first machine learning model is associated with the first user and has been trained to determine a tone for the first user;
continuously training the first machine learning model as the first user interacts with the first device associated with the first user; and
periodically, causing the first machine learning model to be transmitted from the first device associated with the first user.

### Claim 13

One or more non-transitory, computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a voice recording of a meeting, wherein the voice recording comprises a plurality of utterances for a plurality of participants attending the meeting and a corresponding time range for each utterance of the plurality of utterances;
retrieving a list of participants for the meeting;
for each utterance of the plurality of utterances, extracting, using the corresponding time range, corresponding audio associated with each utterance;
comparing each utterance with voice prints of participants in the list of participants;
matching, based on comparing each utterance with the voice prints of the participants in the list of participants, each utterance with a corresponding participant; and
generating a data structure associated with the meeting to include an identifier of the corresponding participant.

### Claim 14 (depends on 13)

The one or more non-transitory, computer-readable media of claim 13, wherein the instructions further cause the one or more processors to perform operations comprising:
generating, using a speech recognition algorithm, a textual transcript of the voice recording, wherein the textual transcript comprises corresponding text of each utterance and the corresponding time range within the voice recording for each utterance; and
adding, to the data structure, a plurality of fields storing for each utterance the corresponding text and the corresponding time range.

### Claim 15 (depends on 13)

The one or more non-transitory, computer-readable media of claim 13, wherein the instructions further cause the one or more processors to perform operations comprising:
generating, from a portion of a recording associated with a first utterance of the plurality of utterances, a plurality of voice characteristics for the first utterance, the plurality of voice characteristics comprising an amplitude and a frequency;
generating an input vector for the first utterance, the input vector comprising the amplitude as the frequency; and
assigning the input vector to utterance data associated with the first utterance.

### Claim 16 (depends on 13)

The one or more non-transitory, computer-readable media of claim 13, wherein the instructions further cause the one or more processors to perform operations comprising:
converting a textual representation of a first utterance of the plurality of utterances into a first vector representation;
converting an audio data associated with the first utterance into a second vector representation; and
assigning the first vector representation and the second vector representation to utterance data associated with the first utterance.

### Claim 17 (depends on 13)

The one or more non-transitory, computer-readable media of claim 13, wherein the instructions further cause the one or more processors to perform operations comprising:
receiving a textual transcript of the meeting, wherein the textual transcript comprises the plurality of utterances for a plurality of users attending the meeting;
selecting a first utterance of the plurality of utterances;
determining, for the first utterance of the plurality of utterances, a user of the plurality of users associated with the first utterance;
selecting, from a plurality of machine learning models associated with the plurality of users, a first machine learning model associated with the user, wherein each machine learning model was trained, using a training set of utterances of a corresponding user, to determine a corresponding tone associated with a particular utterance;
inputting utterance data associated with the first utterance into the first machine learning model to obtain a tone associated with the first utterance, wherein the utterance data comprises voice characteristics associated with the first utterance and textual data of the first utterance; and
modifying a contextual transcript comprising an entry for the first utterance with the tone associated with the first utterance.

### Claim 18 (depends on 17)

The one or more non-transitory, computer-readable media of claim 17, wherein the instructions for selecting the first machine learning model associated with the user further cause the one or more processors to perform operations comprising:
retrieving a first device address associated with a first device of a first user;
transmitting, to the first device address, a command to the first device of the first user, wherein the command instructs the first device of the user to transmit a first tonal machine learning model, wherein the first tonal machine learning model is associated with the first user; and
receiving the first tonal machine learning model from the first device.

### Claim 19 (depends on 13)

The one or more non-transitory, computer-readable media of claim 13, wherein the instructions further cause the one or more processors to perform operations comprising:
storing a first machine learning model on a first device associated with a first user, wherein the first machine learning model is associated with the first user and has been trained to determine a tone for the first user;
continuously training the first machine learning model as the first user interacts with the first device associated with the first user; and
periodically, causing the first machine learning model to be transmitted from the first device associated with the first user.

### Claim 20 (depends on 13)

The one or more non-transitory, computer-readable media of claim 13, wherein the instructions further cause the one or more processors to store the identifier of the corresponding participant, a corresponding time period and a textual representation of a corresponding utterance.