# US 11,451,593 B2 · Persistent co-presence group videoconferencing system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,451,593 B2
- Document ID: US11451593B2
- Status: Granted
- Kind: B2
- Application number: 17/354,544
- Filed: 2021-06-22
- Published: 2022-03-10
- Granted: 2022-09-20
- Earliest claimed: 2020-09-09
- Assignees: Meta Platforms, Inc.
- Applicants: Meta Platforms, Inc.
- Inventors: Ilona Papava; Aryeh Selekman; Nicholas John Shearer; Andreea Selagea
- Web page: https://patentgrep.com/patent/US11451593B2

## CPC

- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44)
- H04L 12/1822: {Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission}
- H04L 12/1827: {Network arrangements for conference optimisation or adaptation}
- H04L 51/043: using or handling presence information
- H04L 65/1069: Session establishment or de-establishment
- H04L 65/1093: by adding participants; by removing participants
- H04L 65/4015: {where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference}
- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- H04M 3/564: {whereby the feature is a sub-conference}
- H04M 3/567: {Multimedia conference systems}
- H04N 7/15: Conference systems
- G06V 40/16: Human faces, e.g. facial parts, sketches or expressions
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)

## Abstract

Disclosed is a system for connecting to an online meeting room with multiple people. A first client device of a first user initiates a connection to the online meeting room. After the connection has been initiated, the first client device starts receiving video data and audio data from one or more client devices of one or more other users connected to the online meeting room. While connected to the online meeting room, the first client device displays a video stream based on the received video data from the one or more client device of the one or more other users connected to the online meeting. Upon receiving a request from a second user to start a conversation with the first user, the first client device plays an audio stream based on the audio data received from the client device of the second user.

## Claims

### Claim 1

A method comprising:
at a first client device associated with a first user:
initiating a connection to an online meeting room,
receiving video data and audio data from one or more client devices of one or more other users also connected to the online meeting room, the one or more client devices of the one or more other users including a second client device associated with a second user,
responsive to being connected to the online meeting room, displaying a video stream based on the received video data from the one or more client devices of the one or more other users connected to the online meeting,
while receiving the video data and audio data from the one or more client devices of the one or more other users connected to the online meeting room, receiving a request from the second client device of the one or more client devices to start a conversation with the first user, and
responsive to receiving the request, playing an audio stream based on the audio data received from the second client device;
for each other user of the one or more other users connected to the online meeting room:
assigning an attenuation factor corresponding to the other user based on a conversation status between the other user and the first user, and
attenuating, based on the assigned attenuation factor, an audio feed corresponding to the other user;
combining the attenuated audio feeds corresponding to each other user of the one or more other users connected to the online meeting room; and
playing the combined attenuated audio feed.

### Claim 2 (depends on 1)

The method of claim 1, wherein displaying the video stream based on the received video data from the one or more client devices of the one or more other users connected to the online meeting comprises:
for each other user of the one or more other users connected to the online meeting room:
detecting a presence of a person in the video data received from a respective client device of the other user, and
responsive to detecting the presence of the person in the video data received from the respective client device of the other user, displaying a video feed corresponding to the other user based on the video data received from the respective client device of the other user.

### Claim 3 (depends on 2)

The method of claim 2, wherein displaying the video stream based on the received video data from the one or more client devices of the one or more other users connected to the online meeting further comprises:
for each other user of the one or more other users connected to the online meeting room:
responsive to not detecting the presence of the person in the video data received from the respective client device of the other user, displaying an indication that the other user is away.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
for each other user of the one or more other users connected to the online meeting room:
determining whether to play the audio feed corresponding to the other user based on the conversation status between the other user and the first user.

### Claim 5 (depends on 1)

The method of claim 1, wherein assigning the attenuation factor corresponding to the other user based on the conversation status between the other user and the first user comprises:
assigning a value of 0 to the attenuation factor when the other user is not in an active conversation with the first user; and
assigning a value of 1 to the attenuation factor when the other user is in an active conversation with the first user.

### Claim 6 (depends on 1)

The method of claim 1, wherein initiating the connection to the online meeting room comprises:
automatically connecting to the online meeting room at a preset time.

### Claim 7 (depends on 1)

The method of claim 1, whether the one or more other users connected to the online meeting room comprise a third user and a fourth user, wherein the method further comprises:
responsive to receiving an indication that the third user is in an active conversation with the fourth user:
displaying the indication that the third user is in the active conversation with the fourth user, and
displaying the indication that the fourth user is in the active conversation with the third user.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
responsive to determining that the first user is not in an active conversation with the third user or the fourth user, muting an audio feed corresponding to the third user and the fourth user.

### Claim 9 (depends on 8)

The method of claim 8, further comprising:
receiving an indication to join the conversation between the third user and the fourth user;
responsive to receiving the indication to join the conversation between the third user and the fourth user:
sending a request to a client device of the third user and a client device of the fourth user to join the conversation, and
playing an audio stream based on the audio data received from the client device of the third user and the client device of the fourth user.

### Claim 10

A method comprising:
receiving a data stream from a plurality of client devices connected to an online meeting room, wherein (i) the plurality of client devices is associated with a plurality of users, (ii) the data stream comprises audio data and status data, and (iii) the status data indicates whether a corresponding user is in an active conversation with another user connected to the online meeting room;
for each client device of the plurality of client devices, determining whether a corresponding user is in an active conversation with another user connected to the online meeting room based on the status data included in the data stream of the user;
responsive to determining that a first user of the plurality of users is in an active conversation with a second user of the plurality of users:
displaying an indication that the first user is in an active conversation with the second user, and
displaying an indication that the second user is in an active conversation with the first user;
receiving an indication to join the conversation between the first user and the second user;
responsive to receiving the indication to join the conversation between the first user and the second user:
sending a request to a client device of the first user and a client device of the second user to join the conversation, and
playing the audio stream corresponding to the first user and the second user.

### Claim 11 (depends on 10)

The method of claim 10, wherein playing the audio stream corresponding to the first user and the second user comprises:
assigning an attenuation factor to each user of the plurality of users connected to the online meeting room, wherein the attenuation factor assigned to the first user and the second user has a non-zero value and the attenuation factor assigned to every other user of the plurality of users connected to the online meeting room has a value of zero and;
attenuating, based on the assigned attenuation factor, an audio feed corresponding to each user of the plurality of users connected to the online meeting room;
generating an audio stream by combining the attenuated audio feeds corresponding to each user of the plurality of users connected to the online meeting room; and
playing the generated audio stream.

### Claim 12 (depends on 10)

The method of claim 10, further comprising:
receiving video data from the client device of each user of the plurality of users connected to the online meeting room; and
responsive to being connected to the online meeting room, displaying a video stream based on the received video data from the client device of each user of the plurality of users connected to the online meeting.

### Claim 13 (depends on 12)

The method of claim 12, wherein displaying the video stream based on the received video data from the client device of each user of the plurality of users connected to the online meeting comprises:
for each user of the plurality of users connected to the online meeting room:
detecting a presence of a person in the video data received from the client device of the user, and
responsive to detecting the presence of the person in the video data received from the client device of the user, displaying a video feed corresponding to the user based on the video data received from the client device of the user.

### Claim 14 (depends on 13)

The method of claim 13, wherein displaying the video stream based on the received video data from the client device of each user of the plurality of users connected to the online meeting further comprises:
for each user of the plurality of users connected to the online meeting room:
responsive to not detecting the presence of a person in the video data received from the client device of the user, displaying an indication that the user is away.

### Claim 15

A non-transitory computer-readable storage medium, storing instructions configured for execution by one or more processors of a first client device associated with a first user, the instructions when executed by the one or more processors cause the first client device to:
initiate a connection to an online meeting room;
receive video data and audio data from one or more client devices of one or more other users also connected to the online meeting room, the one or more client devices of the one or more other users including a second client device associated with a second user;
responsive to being connected to the online meeting room, display a video stream based on the received video data from the one or more client devices of the one or more other users connected to the online meeting;
while receiving the video data and audio data from the one or more client devices of the one or more other users connected to the online meeting room, receive a request from the second client device of the one or more client devices to start a conversation with the first user;
responsive to receiving the request, play an audio stream based on the audio data received from the second client device;
for each other user of the one or more other users connected to the online meeting room:
assign an attenuation factor corresponding to the other user based on a conversation status between the other user and the first user, and
attenuate, based on the assigned attenuation factor, an audio feed corresponding to the other user;
combine the attenuated audio feeds corresponding to each other user of the one or more other users connected to the online meeting room; and
play the combined attenuated audio feed.

### Claim 16 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the instructions to display the video stream based on the received video data from the one or more client devices of the one or more other users connected to the online meeting cause the first client device to:
for each other user of the one or more other users connected to the online meeting room:
detect a presence of a person in the video data received from a respective client device of the other user, and
responsive to detecting the presence of the person in the video data received from the respective client device of the other user, display a video feed corresponding to the other user based on the video data received from the respective client device of the other user.

### Claim 17 (depends on 16)

The non-transitory computer readable storage medium of claim 16, wherein the instructions to display the video stream based on the received video data from the one or more client devices of the one or more other users connected to the online meeting further cause the first client device to:
for each other user of the one or more other users connected to the online meeting room:
responsive to not detecting the presence of the person in the video data received from the respective client device of the other user, display an indication that the other user is away.

### Claim 18 (depends on 15)

The non-transitory computer readable storage medium of claim 15, wherein the instructions further cause the first client device to:
for each other user of the one or more other users connected to the online meeting room:
determine whether to play the audio feed corresponding to the other user based on the conversation status between the other user and the first user.