# US 11,909,784 B2 · Automated actions in a conferencing service

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,909,784 B2
- Document ID: US11909784B2
- Status: Granted
- Kind: B2
- Application number: 17/521,900
- Filed: 2021-11-09
- Published: 2023-02-02
- Granted: 2024-02-20
- Earliest claimed: 2021-07-29
- Assignees: VMware, Inc.
- Applicants: VMware LLC
- Inventors: Rohit Pradeep Shetty; Ravish Chawla; Adam Chow
- Web page: https://patentgrep.com/patent/US11909784B2

## CPC

- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path}
- G06F 40/166: Editing, e.g. inserting or deleting
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G10L 15/005: {Language recognition}
- G10L 15/18: using natural language modelling
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 25/57: for processing of video signals
- G06F 40/263: Language identification
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- H04L 65/1089: by adding media; by removing media
- H04L 65/1093: by adding participants; by removing participants
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Disclosed are various approaches for performing automated actions in a conferencing service. Distractions can be detected and users can be muted. Breakout rooms can be suggested to attendees based upon the user's identity. Additionally, event summaries and recaps can be generated for users who are late-arriving or who depart and return to the event.

## Claims

### Claim 1

A method comprising:
identifying an event in at least one of a user calendar or a conferencing service, the event having a plurality of attendees;
identifying a primary language associated with the event based upon an identity of the attendees;
obtaining an audio component and a video component corresponding to the event;
analyzing the audio component and the video component to identify at least one of a visual distraction or an audio distraction, wherein the visual distraction or the audio distraction is associated with a beginning;
identifying an attendee of the event associated with the at least one of a visual distraction or an audio distraction by detecting a spoken language in a portion of the audio component that is spoken by the attendee during the event and obtained from the attendee that is different from the primary language associated with the event, wherein the portion of the audio component originates from a client associated with the attendee;
performing a remedial action with respect to the attendee in response to identifying the at least one of the visual distraction or the audio distraction;
detect an end of the visual distraction or the audio distraction; and
generate a summary of the event from the beginning and the end, the summary generated based upon spoken content and video content using a natural language processing (NLP) model applied to the spoken content from the beginning and the end of the visual distraction and the audio distraction.

### Claim 2 (depends on 1)

The method of claim 1, wherein the remedial action comprises at least one of: muting the attendee, adjusting a volume associated with the attendee, or generating a suggestion for the attendee to be placed into a breakout room within the conferencing service.

### Claim 3 (depends on 1)

The method of claim 1, wherein generating the suggestion for at least one breakout room is based upon an identity of the attendees, the suggestion communicated to the attendees using an attendee chat feature associated with the conferencing service.

### Claim 4 (depends on 3)

The method of claim 3, wherein the suggestion comprises a first breakout room for a first subset of attendees associated with a first user group and a second breakout room for a second subset of attendees associated with a second user group.

### Claim 5 (depends on 1)

The method of claim 1, further comprising detecting a conclusion of the event and generating a text summary of the event based upon a natural language processing process performed on the audio component, wherein the summary of the event is provided to attendees of the meeting through a conferencing service client.

### Claim 6

A system, comprising:
a computing device comprising at least one processor and at least one memory; and
machine-readable instructions stored in the at least one memory, wherein the instructions, when executed by the at least one processor, cause the computing device to at least:
identify an event in at least one of a user calendar or a conferencing service, the event having a plurality of attendees;
identify a primary language associated with the event based upon an identity of the attendees;
obtain an audio component and a video component corresponding to the event;
analyze the audio component and the video component to identify at least one of a visual distraction or an audio distraction, wherein the visual distraction or the audio distraction is associated with a beginning;
identify an attendee of the event associated with the at least one of a visual distraction or an audio distraction by detecting a spoken language in a portion of the audio component that is spoken by the attendee during the event and obtained from the attendee that is different from the primary language associated with the event, wherein the portion of the audio component originates from a client associated with the attendee;
perform a remedial action with respect to the attendee in response to identifying the at least one of the visual distraction or the audio distraction;
detect an end of the visual distraction or the audio distraction; and
generate a summary of the event from the beginning and the end, the summary generated based upon spoken content and video content using a natural language processing (NLP) model applied to the spoken content from the beginning and the end of the visual distraction and the audio distraction.

### Claim 7 (depends on 6)

The system of claim 6, wherein the remedial action comprises at least one of: muting the attendee, adjusting a volume associated with the attendee, or generating a suggestion for the attendee to be placed into a breakout room within the conferencing service.

### Claim 8 (depends on 6)

The system of claim 6, wherein the machine-readable instructions generate a suggestion for at least one breakout room based upon an identity of the attendees, the suggestion communicated to the attendees using an attendee chat feature associated with the conferencing service.

### Claim 9 (depends on 8)

The system of claim 8, wherein the suggestion comprises a first breakout room for a first subset of attendees associated with a first user group and a second breakout room for a second subset of attendees associated with a second user group.

### Claim 10 (depends on 6)

The system of claim 6, wherein the machine readable instructions further cause the at least one computing device to at least detect entry into the event by an attendee after commencement of the event and generate a summary of a previous time period associated with the event.

### Claim 11 (depends on 6)

The system of claim 6, wherein the machine readable instructions further cause the at least one computing device to at least detect a conclusion of the event and generate a text summary of the event based upon a natural language processing process performed on the audio component, wherein the summary of the event is provided to attendees of the meeting through a conferencing service client.

### Claim 12

A non-transitory computer-readable medium comprising machine-readable instructions, wherein the instructions, when executed by at least one processor, cause a computing device to at least:
identify an event in at least one of a user calendar or a conferencing service, the event having a plurality of attendees;
identify a primary language associated with the event based upon an identity of the attendees;
obtain an audio component and a video component corresponding to the event;
analyze the audio component and the video component to identify at least one of a visual distraction or an audio distraction, wherein the visual distraction or the audio distraction is associated with a beginning;
identify an attendee of the event associated with the at least one of a visual distraction or an audio distraction by detecting a spoken language in a portion of the audio component that is spoken by the attendee during the event and obtained from the attendee that is different from the primary language associated with the event, wherein the portion of the audio component originates from a client associated with the attendee;
perform a remedial action with respect to the attendee in response to identifying the at least one of the visual distraction or the audio distraction;
detect an end of the visual distraction or the audio distraction; and
generate a summary of the event from the beginning and the end, the summary generated based upon spoken content and video content using a natural language processing (NLP) model applied to the spoken content from the beginning and the end of the visual distraction and the audio distraction.

### Claim 13 (depends on 12)

The non-transitory computer-readable medium of claim 12, wherein the remedial action comprises at least one of: muting the attendee, adjusting a volume associated with the attendee, or generating a suggestion for the attendee to be placed into a breakout room within the conferencing service.

### Claim 14 (depends on 12)

The non-transitory computer-readable medium of claim 12, wherein the machine-readable instructions generate a suggestion for at least one breakout room based upon an identity of the attendees, the suggestion communicated to the attendees using an attendee chat feature associated with the conferencing service.

### Claim 15 (depends on 14)

The non-transitory computer-readable medium of claim 14, wherein the suggestion comprises a first breakout room for a first subset of attendees associated with a first user group and a second breakout room for a second subset of attendees associated with a second user group.

### Claim 16 (depends on 12)

The non-transitory computer-readable medium of claim 12, wherein the instructions detect entry into the event by an attendee after commencement of the event and generate a summary of a previous time period associated with the event.

### Claim 17 (depends on 12)

The non-transitory computer-readable medium of claim 12, wherein the instructions further cause the at least one computing device to at least detect a conclusion of the event and generate a text summary of the event based upon a natural language processing process performed on the audio component, wherein the summary of the event is provided to attendees of the meeting through a conferencing service client.