# US 12,182,500 B2 · Generating meeting notes

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,182,500 B2
- Document ID: US12182500B2
- Status: Granted
- Kind: B2
- Application number: 18/333,922
- Filed: 2023-06-13
- Published: 2024-02-01
- Granted: 2024-12-31
- Earliest claimed: 2022-07-31
- Assignees: Zoom Video Communications, Inc.
- Applicants: Zoom Video Communications, Inc.
- Inventors: Ravi Teja Polavaram
- Web page: https://patentgrep.com/patent/US12182500B2

## CPC

- G06T 11/60: Creating or editing images; Combining images with text
- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06F 40/166: Editing, e.g. inserting or deleting
- G06F 40/216: using statistical methods
- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06F 40/30: Semantic analysis
- G06F 40/35: Discourse or dialogue representation
- G06Q 10/10: Office automation; Time management
- G06Q 10/101: Collaborative creation, e.g. joint development of products or services
- H04L 65/1069: Session establishment or de-establishment
- H04L 65/1089: by adding media; by removing media
- H04L 65/4015: {where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Methods and systems provide for real-time meeting notes within a communication session. In one embodiment, the system connects to a communication session with a plurality of participants; receives, in real time during the communication session, updates to a transcript of a conversation between the participants produced during the communication session, the transcript including timestamps for a plurality of utterances associated with speaking participants; receives a request from one of the participants to generate a meeting note which includes a subset of utterances from the transcript; determines a starting timestamp and an ending timestamp for utterances representing the boundaries of the meeting note; generates, in real time during the communication session, a meeting note where the utterances for the starting timestamp and ending timestamp are the starting utterance and ending utterance of the meeting note, respectively; and presents to at least the participant, within a user interface, the generated meeting note.

## Claims

### Claim 1

A method, comprising:
receiving a request to generate a meeting note corresponding to a first timestamp for a meeting with one or more participants, wherein the request is initiated by a participant of the one or more participants at the first timestamp;
determining, based on a transcript of the meeting, a context of the meeting at the first timestamp according to contexts of utterances prior to the first timestamp and contexts of utterances subsequent to the first timestamp, the determining being performed by an artificial intelligence (AI) model trained via self-supervised pre-training;
determining a starting timestamp and an ending timestamp representing boundaries of the meeting note that correspond to the context of the meeting at the first timestamp;
generating the meeting note according to the starting timestamp and the ending timestamp; and
presenting the generated meeting note to the one or more participants.

### Claim 2 (depends on 1)

The method of claim 1, wherein presenting the generated meeting note to the one or more participants occurs in real time during the meeting.

### Claim 3 (depends on 2)

The method of claim 2, wherein the generated meeting note is editable by the participants in real time during the meeting.

### Claim 4 (depends on 1)

The method of claim 1, wherein the generated meeting note is presented after the meeting has terminated within an after-meeting UI.

### Claim 5 (depends on 4)

The method of claim 4, wherein the generated meeting note is presented within the after-meeting UI concurrently with one or more additional meeting notes generated during the meeting.

### Claim 6 (depends on 1)

The method of claim 1, wherein presenting the generated meeting note comprises presenting at least a subset of a transcript of the meeting such that the utterances within the boundaries of the meeting note are highlighted within the transcript.

### Claim 7 (depends on 6)

The method of claim 6, wherein the highlighted utterances within the boundaries of the meeting note are modifiable by a participant to adjust one or more of the starting timestamp and ending timestamp.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
receiving a request from a participant to edit one or more of the starting timestamp and the ending timestamp; and
determining, based on the request, one or more of a new starting timestamp and a new ending timestamp.

### Claim 9 (depends on 8)

The method of claim 8, wherein the new starting timestamp and new ending timestamp are ingested as training data for training a model to determine starting timestamps and ending timestamps for one or more future meeting notes.

### Claim 10 (depends on 1)

The method of claim 1, wherein the AI model is trained to: extract one or more content-based features; and
generate a prediction of the starting timestamp and the ending timestamp based at least in part on the one or more content-based features.

### Claim 11 (depends on 10)

The method of claim 10, wherein the AI model extracts the content-based features using one or more of: a TextTiling training model, and a language model via self-supervised pre-training.

### Claim 12 (depends on 1)

The method of claim 1, wherein the AI model is trained to: extract one or more multimodal features; and
generate a prediction of the starting timestamp and the ending timestamp based at least in part on the one or more multimodal features.

### Claim 13

A communication system comprising one or more processors configured to perform operations:
receiving a request to generate a meeting note corresponding to a first timestamp for a meeting with one or more participants, wherein the request is initiated by a participant of the one or more participants at the first timestamp;
determining, based on a transcript of the meeting, a context of the meeting at the first timestamp according to contexts of utterances prior to the first timestamp and contexts of utterances subsequent to the first timestamp, the determining being performed by an artificial intelligence (AI) model trained via self-supervised pre-training;
determining a starting timestamp and an ending timestamp representing boundaries of the meeting note that correspond to the context of the meeting at the first timestamp;
generating the meeting note according to the starting timestamp and the ending timestamp; and
presenting the generated meeting note to the one or more participants.

### Claim 14 (depends on 13)

The communication system of claim 13, wherein determining the starting timestamp and the ending timestamp for the meeting note comprises:
extracting one or more conversational features via a rule-based model; and
determining the starting timestamp and the ending timestamp based at least in part on the conversational features.

### Claim 15 (depends on 13)

The communication system of claim 13, wherein determining the starting timestamp and the ending timestamp for the meeting note comprises employing a pairwise meeting note boundary to generate a list of candidates via a rule-based model, each candidate comprising a boundary pair of a starting utterance and an ending utterance representing a boundary for the meeting note.

### Claim 16 (depends on 15)

The communication system of claim 15, wherein determining the starting timestamp and the ending timestamp for the meeting note further comprises employing the pairwise meeting note boundary to generate rankings for the each boundary pairs via a random forest regressor.

### Claim 17 (depends on 13)

The communication system of claim 13, wherein the request to generate a meeting note is initiated by a participant by interactively selecting a UI element for saving a note.

### Claim 18 (depends on 13)

The communication system of claim 13, wherein the one or more processors are further configured to perform the operations of:
detecting one or more automated note taking conditions,
wherein the request to generate a meeting note is automatically initiated based on the detecting of the one or more automated note taking conditions.

### Claim 19

A non-transitory computer-readable medium containing instructions comprising:
instructions for receiving a request to generate a meeting note corresponding to a first timestamp for a meeting with one or more participants, wherein the request is initiated by a participant of the one or more participants at the first timestamp;
instructions for determining, based on a transcript of the meeting, a context of the meeting at the first timestamp according to contexts of utterances prior to the first timestamp and contexts of utterances subsequent to the first timestamp, the determining being performed by an artificial intelligence (AI) model trained via self-supervised pre-training;
instructions for determining a starting timestamp and an ending timestamp representing boundaries of the meeting note that correspond to the context of the meeting at the first timestamp;
instructions for generating the meeting note according to the starting timestamp and the ending timestamp; and
instructions for presenting the generated meeting note to the one or more participants.

### Claim 20 (depends on 19)

The non-transitory computer readable medium of claim 19, further containing instructions comprising:
instructions for receiving a request from a participant to edit one or more of the starting timestamp and the ending timestamp; and
instructions for determining, based on the request, one or more of a new starting timestamp and a new ending timestamp.