# US 12,192,017 B2 · Advanced routing and remote rendered videos layout for meeting side group conversations and voice assistant providers > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,017 B2 - Document ID: US12192017B2 - Status: Granted - Kind: B2 - Application number: 18/087,308 - Filed: 2022-12-22 - Published: 2024-06-27 - Granted: 2025-01-07 - Earliest claimed: 2022-12-22 - Assignees: Adeia Guides Inc. - Applicants: Adeia Guides Inc. - Inventors: Christopher Phillips; Reda Harb; Tao Chen - Web page: https://patentgrep.com/patent/US12192017B2 ## CPC - H04L 12/1822: {Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission} - H04L 65/1093: by adding participants; by removing participants - H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Systems and methods for routing audio and/or video streams and remote rendered video layout for conference side group conversations and voice assistant providers are disclosed. During a main conference comprising a plurality of users, a request is received to create a side group conversation comprising a subgroup of the plurality of users. A voice assistant provider is invited to the side group conversation. Users in the subgroup may be granted different permissions to query or receive responses from the voice assistant. Incoming audio and video streams from each user are demultiplexed and decoded. Decoded audio streams are processed by an appropriate audio mixer of the group and voice assistant. Decoded video streams are processed and rendered based on a video grouping render policies. Processed audio streams for each group and voice assistant and/or rendered videos are encoded and routed to each user through their dedicated multiplexer. ## Claims ### Claim 1 A method comprising: providing, by a server, a conference session, wherein a plurality of participants attends the conference session; creating, by the server, a side conversation from the conference session; inviting, by the server, a subgroup of the plurality of participants to the side conversation; receiving a first request, by the server from a first participant of the subgroup of the plurality of participants, to invite a first voice assistant associated with a first voice assistant service provider to the side conversation; receiving a second request, by the server from a second participant of the subgroup of the plurality of participants, to invite to the side conversation a second voice assistant associated with a second voice assistant service provider different from the first voice assistant service provider; and adding to the side conversation, by the server, the first voice assistant associated with the first voice assistant service provider and the second voice assistant associated with the second voice assistant service provider different from the first voice assistant service provider. ### Claim 2 (depends on 1) The method of claim 1, further comprising: routing audio and/or video streams associated with the conference session between the plurality of participants; and at substantially the same time, routing audio and/or video streams associated with the side conversation between the subgroup of the plurality of participants. ### Claim 3 (depends on 2) The method of claim 2, further comprising: routing audio and/or video streams associated with the first voice assistant between the subgroup of the plurality of participants and the first voice assistant based on a voice assistant permissions policy. ### Claim 4 (depends on 3) The method of claim 3, wherein the voice assistant permissions policy comprises permitting at least one of the subgroup to at least query the first voice assistant or receive responses from the first voice assistant. ### Claim 5 (depends on 1) The method of claim 1, further comprising: creating a second side conversation from the conference session; and inviting a second subgroup of the plurality of participants to the second side conversation, wherein the second subgroup includes at least one participant from the first-referenced subgroup. ### Claim 6 (depends on 5) The method of claim 5, further comprising: inviting the first voice assistant to the second side conversation. ### Claim 7 (depends on 3) The method of claim 3, further comprising: managing, based on preferences of a client device, output features associated with the audio and/or video streams associated with the conference session, the audio and/or video streams associated with the side conversation, and the audio and/or video streams associated with the first voice assistant. ### Claim 8 (depends on 1) The method of claim 1, further comprising: configuring a video grouping for rendering at a client device, wherein the video grouping comprises a list of participants sorted by group conversations. ### Claim 9 (depends on 1) The method of claim 1, further comprising: enabling a participant to speak in the conference session while participating in the side conversation. ### Claim 10 A system comprising: memory; control circuitry configured to: provide a conference session stored at least in part in the memory, wherein a plurality of participants attends the conference session; create a side conversation from the conference session; invite a subgroup of the plurality of participants to the side conversation; receive a first request, from a first participant of the subgroup of the plurality of participants, to invite a first voice assistant associated with a first voice assistant service provider to the side conversation; and receive a second request, from a second participant of the subgroup of the plurality of participants, to invite to the side conversation a second voice assistant associated with a second voice assistant service provider different from the first voice assistant service provider; and add to the side conversation the first voice assistant associated with the first voice assistant service provider and the second voice assistant associated with the second voice assistant service provider different from the first voice assistant service provider. ### Claim 11 (depends on 10) The system of claim 10, wherein the control circuitry is further configured to: route audio and/or video streams associated with the conference session between the plurality of participants; and at substantially the same time, route audio and/or video streams associated with the side conversation between the subgroup of the plurality of participants. ### Claim 12 (depends on 11) The system of claim 11, wherein the control circuitry is further configured to: route audio and/or video streams associated with the first voice assistant between the subgroup of the plurality of participants and the first voice assistant based on a voice assistant permissions policy. ### Claim 13 (depends on 12) The system of claim 12, wherein the first voice assistant permissions policy comprises permitting at least one of the subgroup to at least query the first voice assistant or receive responses from the first voice assistant. ### Claim 14 (depends on 10) The system of claim 10, wherein the control circuitry is further configured to: create a second side conversation from the conference session; and invite a second subgroup of the plurality of participants to the second side conversation, wherein the second subgroup includes at least one participant from the first-referenced subgroup. ### Claim 15 (depends on 14) The system of claim 14, wherein the control circuitry is further configured to invite the first voice assistant to the second side conversation. ### Claim 16 (depends on 12) The system of claim 12, wherein the control circuitry is further configured to: manage, based on preferences of a client device, output features associated with the audio and/or video streams associated with the conference session, the audio and/or video streams associated with the side conversation, and the audio and/or video streams associated with the first voice assistant. ### Claim 17 (depends on 10) The system of claim 10, wherein the control circuitry is further configured to: configure a video grouping for rendering at a client device, wherein the video grouping comprises a list of participants sorted by group conversations. ### Claim 18 (depends on 10) The system of claim 10, wherein the control circuitry is further configured to: enable a participant to speak in the conference session while participating in the side conversation.