# US 2023/0402038 A1 · Computerized intelligent assistant for conferences

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0402038 A1
- Document ID: US20230402038A1
- Status: Published application
- Kind: A1
- Application number: 18/197,595
- Filed: 2023-05-15
- Published: 2023-12-14
- Earliest claimed: 2018-05-04
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Adi DIAMANT; Xuedong HUANG; Karen MASTER BEN-DOR; Eyal KRUPKA; Raz HALALY; Yoni SMOLIN; Ilya GURVICH; Aviv HURVITZ; Lijuan QIN; Wei XIONG; Shixiong ZHANG; Lingfeng WU; Xiong XIAO; Ido LEICHTER; Moshe DAVID; Amit Kumar AGARWAL
- Web page: https://patentgrep.com/patent/US20230402038A1

## CPC

- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- G06V 40/172: {Classification, e.g. identification}
- G10L 17/00: Speaker identification or verification techniques
- H04N 7/15: Conference systems
- H04N 7/155: {involving storage of or access to video conference sessions (tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831)}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for facilitating a remote conference includes receiving a digital video and a computer-readable audio signal. A face recognition machine is operated to recognize a face of a first conference participant in the digital video, and a speech recognition machine is operated to translate the computer-readable audio signal into a first text. An attribution machine attributes the text to the first conference participant. A second computer-readable audio signal is processed similarly, to obtain a second text attributed to a second conference participant. A transcription machine automatically creates a transcript including the first text attributed to the first conference participant and the second text attributed to the second conference participant.

## Claims

### Claim 1

At least one computer-storage device embodying computer-usable instructions which, when executed by at least one processor, implement a method comprising:
obtaining information identifying a participant of a meeting;
generating a transcript of the meeting by at least converting audio of a conversation by the participant to text;
tracking an event of interest to the participant occurring during the meeting by at least generating a reviewable transcript based on the transcript and the event of interest; and
providing a notification of the reviewable transcript to the participant.

### Claim 2 (depends on 1)

The at least one computer-storage device of claim 1, wherein the event of interest is recorded in the reviewable transcript and includes at least one of: arrivals of participants, departures of participants, video captured during the meeting, visual information shared by participants, digital information shared by participants, data generated by a companion device, gestures performed by participant, and interactions by the participants.

### Claim 3 (depends on 1)

The at least one computer-storage device of claim 1, wherein the reviewable transcript includes information correlating the evens of interest with the transcript based on timestamps included in the transcript.

### Claim 4 (depends on 1)

The at least one computer-storage device of claim 1, wherein the notification of the reviewable transcript to the participant is provided to the participant during the meeting.

### Claim 5 (depends on 1)

The at least one computer-storage device of claim 1, wherein the notification is provided to a companion device associated with the participant.

### Claim 6 (depends on 1)

The at least one computer-storage device of claim 1, wherein the reviewable transcript includes a digital image or a video captured during the meeting corresponding to the event of interest.

### Claim 7 (depends on 1)

The at least one computer-storage device of claim 1, wherein the reviewable transcript includes a set of transcript entries indicating information associated with the event of interest, at least one transcript entry indicating an arrival of the participant to the meeting.

### Claim 8 (depends on 7)

The at least one computer-storage device of claim 7, comprising determining the arrival of the participant based on at least one of: a signature associated with the participant; a set of users invited to the meeting, audio capture during the meeting, video captured during the meeting, and natural language features of the transcript.

### Claim 9

A system comprising:
at least one processor;
at least one storage device storing computer-usable instructions which, when executed by the at least one processor, implement operations comprising:
determining a meeting including a participant has started;
identifying the participant;
generating a reviewable transcript including information relevant to the participant by at least:
converting audio of the meeting to text; and
detecting a event of the meeting; and
providing a notification of the reviewable transcript to the participant.

### Claim 10 (depends on 9)

The system of claim 9, wherein the event includes an arrival of a second participant to the meeting.

### Claim 11 (depends on 10)

The system of claim 10, the operations further comprising detecting the arrival of the second participant, wherein generating the reviewable transcript including the information relevant to the participant is performed in response to detecting the arrival of the second participant.

### Claim 12 (depends on 9)

The system of claim 9, wherein the information relevant to the participant includes information recorded during an interval of time during the meeting when the first participant was absent based on an arrival of the participant to the meeting.

### Claim 13 (depends on 9)

The system of claim 9, wherein identifying the participant further comprises identifying the participant based on the participant being at least one of: an invited participant, a colleague of the invited participant, and an individual with access to location the meeting is being held.

### Claim 14 (depends on 9)

The system of claim 9, wherein identifying the participant further comprises prompting the participant to provide further identifying information by at least providing a question to a companion device associated with the participant.

### Claim 15 (depends on 9)

The system of claim 9, the operations further comprising collecting, in response to identifying the participant, audio information or video information associated with the participant to update a signature associated with the participant, where the signature is used to identify the participant.

### Claim 16

A method comprising:
determining a participant of a meeting;
detecting an event occurring during the meeting relevant to the participant;
generating a reviewable transcript for the participant including text converted from audio of the meeting and the event; and
providing the reviewable transcript to the participant.

### Claim 17 (depends on 16)

The method of claim 16, wherein the method further comprises generating a second reviewable transcript for a second participant including a second event that is distinct from the event.

### Claim 18 (depends on 16)

The method of claim 16, wherein the reviewable transcript further comprises a sentiment analysis associated with the participant and indicating sentiment-related words obtained from the text converted from the audio of the meeting.

### Claim 19 (depends on 16)

The method of claim 16, wherein the reviewable transcript further comprises a participation summary associated with the participant indicating the event is attributable to the participant.

### Claim 20 (depends on 16)

The method of claim 16, wherein an event of the subset of events includes a task assigned to the participant during the meeting.
