# US 11,444,988 B2 · Signaling of scene description for multimedia conferencing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,444,988 B2 - Document ID: US11444988B2 - Status: Granted - Kind: B2 - Application number: 17/069,423 - Filed: 2020-10-13 - Published: 2021-09-23 - Granted: 2022-09-13 - Earliest claimed: 2020-03-17 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Imed Bouazizi; Nikolai Konrad Leung; Thomas Stockhammer - Web page: https://patentgrep.com/patent/US11444988B2 ## CPC - H04N 21/235: Processing of additional data, e.g. scrambling of additional data or processing content descriptors - G06F 16/9024: {Graphs; Linked lists (G06F16/9027 takes precedence)} - H04L 65/1069: Session establishment or de-establishment - H04L 65/1104: Session initiation protocol [SIP] - H04L 65/1108: Web based protocols, e.g. webRTC - H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) - H04L 65/765: {intermediate} - H04N 21/23412: {for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects} - H04N 21/44012: {involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs} - H04N 21/6437: Real-time Transport Protocol [RTP] - H04N 21/816: {involving special video data, e.g 3D video} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Various embodiments include systems and methods for providing an immersive three-dimensional group session. Various embodiments include methods and devices for signaling a description of a scene with media components potentially coming from different parties. In various aspects, a scene graph may be signaled through a session description protocol (SDP) during a session initiation protocol (SIP) session setup. In various embodiments, the scene graph may include respective graphical output nodes assigned to be controlled by each of the plurality of participant computing devices in an immersive three-dimensional group session. ## Claims ### Claim 1 A method performed by a processor of a wireless device that is one of a plurality of participant computing devices operating in an immersive three-dimensional group session, comprising: receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates that at least an own graphical output node is controlled by the wireless device and indicates that respective other graphical output nodes are controlled by each of the other of the plurality of participant computing devices; controlling components of the own graphical output node relative to a three-dimensional space of the immersive three-dimensional group session; sending the components of the own graphical output node in a first media stream to the other of the plurality of participant computing devices; receiving components of the other graphical output nodes in media streams from each of the other of the plurality of participant computing devices; and rendering the immersive three-dimensional group session on a display of the wireless device based at least in part on the components of the own graphical output node and the components of the other graphical output nodes. ### Claim 2 (depends on 1) The method of claim 1, further comprising: receiving a scene graph update including an indication of a new participant computing device for the immersive three-dimensional group session and an indication of a new graphical output node controlled by the new participant computing device; receiving components of the new graphical output node in a second media stream from the new participant computing device; and rendering the immersive three-dimensional group session on the display of the wireless device based at least in part on the components of the own graphical output node, the components of the other graphical output nodes, and the components of the new graphical output node. ### Claim 3 (depends on 1) The method of claim 1, further comprising: receiving a session description protocol (SDP) for the immersive three-dimensional group session indicating an address of a data channel over which the scene graph will be shared, wherein receiving the scene graph comprises downloading the scene graph via the data channel. ### Claim 4 (depends on 1) The method of claim 1, further comprising: sending an offer to send or to receive the scene graph to the other of the plurality of participant computing devices as part of session initiation protocol (SIP) setup for the immersive three-dimensional group session. ### Claim 5 (depends on 1) The method of claim 1, further comprising: sending an offer to send or to receive the scene graph to the other of the plurality of participant computing devices, wherein the offer indicates the own graphical output node. ### Claim 6 (depends on 1) The method of claim 1, wherein the immersive three-dimensional group session is a Web Real-Time Communications (WebRTC) session. ### Claim 7 (depends on 1) The method of claim 1, wherein controlling components of the own graphical output node relative to the three-dimensional space of the immersive three-dimensional group session comprises controlling components of the own graphical output node based at least in part on a determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session. ### Claim 8 (depends on 7) The method of claim 7, wherein controlling components of the own graphical output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session comprises controlling components of the own graphical output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session and a determined orientation of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session. ### Claim 9 A wireless device, comprising: a display; and a processor coupled to the display, the processor configured with processor-executable instructions to: receive a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates that at least an own graphical output node is controlled by the wireless device and indicates that respective other graphical output nodes are controlled by each of a plurality of participant computing devices; control components of the own graphical output node relative to a three-dimensional space of the immersive three-dimensional group session; send the components of the own graphical output node in a first media stream to the other of the plurality of participant computing devices; receive components of the other graphical output nodes in media streams from each of the other of the plurality of participant computing devices; and render the immersive three-dimensional group session on the display of the wireless device based at least in part on the components of the own graphical output node and the components of the other graphical output nodes. ### Claim 10 (depends on 9) The wireless device of claim 9, wherein the processor is further configured with processor-executable instructions to: receive a scene graph update including an indication of a new participant computing device for the immersive three-dimensional group session and an indication of a new graphical output node controlled by the new participant computing device; receive components of the new graphical output node in a second media stream from the new participant computing device; and render the immersive three-dimensional group session on the display of the wireless device based at least in part on the components of the own graphical output node, the components of the other graphical output nodes, and the components of the new graphical output node. ### Claim 11 (depends on 9) The wireless device of claim 9, wherein the processor is further configured with processor-executable instructions to: receive a session description protocol (SDP) for the immersive three-dimensional group session indicating an address of a data channel over which the scene graph will be shared, and wherein the processor is further configured with processor-executable instructions to receive the scene graph by downloading the scene graph via the data channel. ### Claim 12 (depends on 9) The wireless device of claim 9, wherein the processor is further configured with processor-executable instructions to: send an offer to send or to receive the scene graph to the other of the plurality of participant computing devices as part of session initiation protocol (SIP) setup for the immersive three-dimensional group session. ### Claim 13 (depends on 9) The wireless device of claim 9, wherein the processor is further configured with processor-executable instructions to: send an offer to send or to receive the scene graph to the other of the plurality of participant computing devices, wherein the offer indicates the own graphical output node. ### Claim 14 (depends on 9) The wireless device of claim 9, wherein the immersive three-dimensional group session is a Web Real-Time Communications (WebRTC) session. ### Claim 15 (depends on 9) The wireless device of claim 9, wherein the processor is further configured with processor-executable instructions to control components of the own graphical output node relative to the three-dimensional space of the immersive three-dimensional group session by controlling components of the own graphical output node based at least in part on a determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session. ### Claim 16 (depends on 15) The wireless device of claim 15, wherein the processor is further configured with processor-executable instructions to control components of the own graphical output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session by controlling components of the own graphical output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session and a determined orientation of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session. ### Claim 17 A non-transitory processor readable medium having stored thereon processor-executable instructions configured to cause a processor of a wireless device to perform operations comprising: receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates that at least an own graphical output node is controlled by the wireless device and indicates that respective other graphical output nodes are controlled by each of a plurality of participant computing devices; controlling components of the own graphical output node relative to a three-dimensional space of the immersive three-dimensional group session; sending the components of the own graphical output node in a first media stream to the other of the plurality of participant computing devices; receiving components of the other graphical output nodes in media streams from each of the other of the plurality of participant computing devices; and rendering the immersive three-dimensional group session on a display of the wireless device based at least in part on the components of the own graphical output node and the components of the other graphical output nodes. ### Claim 18 (depends on 17) The non-transitory processor readable medium of claim 17, wherein the stored processor-executable instructions are configured to cause a processor of a wireless device to perform operations further comprising: receiving a scene graph update including an indication of a new participant computing device for the immersive three-dimensional group session and an indication of a new graphical output node controlled by the new participant computing device; receiving components of the new graphical output node in a second media stream from the new participant computing device; and rendering the immersive three-dimensional group session on the display of the wireless device based at least in part on the components of the own graphical output node, the components of the other graphical output nodes, and the components of the new graphical output node. ### Claim 19 (depends on 17) The non-transitory processor readable medium of claim 17, wherein the stored processor-executable instructions are configured to cause a processor of a wireless device to perform operations further comprising: receiving a session description protocol (SDP) for the immersive three-dimensional group session indicating an address of a data channel over which the scene graph will be shared, and wherein the processor-executable instructions are configured to cause a processor of a wireless device to perform operations such that receiving the scene graph comprises downloading the scene graph via the data channel. ### Claim 20 (depends on 17) The non-transitory processor readable medium of claim 17, wherein the stored processor-executable instructions are configured to cause a processor of a wireless device to perform operations further comprising: sending an offer to send or to receive the scene graph to the other of the plurality of participant computing devices as part of session initiation protocol (SIP) setup for the immersive three-dimensional group session. ### Claim 21 (depends on 17) The non-transitory processor readable medium of claim 17, wherein the stored processor-executable instructions are configured to cause a processor of a wireless device to perform operations further comprising: sending an offer to send or to receive the scene graph to the other of the plurality of participant computing devices, wherein the offer indicates the own graphical output node. ### Claim 22 (depends on 17) The non-transitory processor readable medium of claim 17, wherein the immersive three-dimensional group session is a Web Real-Time Communications (WebRTC) session. ### Claim 23 (depends on 17) The non-transitory processor readable medium of claim 17, wherein the stored processor-executable instructions are configured to cause a processor of a wireless device to perform operations such that controlling components of the own graphical output node relative to the three-dimensional space of the immersive three-dimensional group session comprises controlling components of the own graphical output node based at least in part on a determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session. ### Claim 24 (depends on 17) The non-transitory processor readable medium of claim 17, wherein the stored processor-executable instructions are configured to cause a processor of a wireless device to perform operations such that controlling components of the own graphical output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session comprises controlling components of the own graphical output node based at least in part on the determined position of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session and a determined orientation of the wireless device relative to the three-dimensional space of the immersive three-dimensional group session. ### Claim 25 A wireless device, comprising: means for receiving a scene graph for an immersive three-dimensional group session, wherein the scene graph indicates that at least an own graphical output node is controlled by the wireless device and indicates that respective other graphical output nodes are controlled by each of a plurality of participant computing devices; means for controlling components of the own graphical output node relative to a three-dimensional space of the immersive three-dimensional group session; means for sending the components of the own graphical output node in a first media stream to the other of the plurality of participant computing devices; means for receiving components of the other graphical output nodes in media streams from each of the other of the plurality of participant computing devices; and means for rendering the immersive three-dimensional group session on a display of the wireless device based at least in part on the components of the own graphical output node and the components of the other graphical output nodes. ### Claim 26 (depends on 25) The wireless device of claim 25, further comprising: means for receiving a scene graph update including an indication of a new participant computing device for the immersive three-dimensional group session and an indication of a new graphical output node controlled by the new participant computing device; means for receiving components of the new graphical output node in a second media stream from the new participant computing device; and means for rendering the immersive three-dimensional group session on the display of the wireless device based at least in part on the components of the own graphical output node, the components of the other graphical output nodes, and the components of the new graphical output node. ### Claim 27 (depends on 25) The wireless device of claim 25, further comprising: means for receiving a session description protocol (SDP) for the immersive three-dimensional group session indicating an address of a data channel over which the scene graph will be shared, wherein means for receiving the scene graph comprises means for downloading the scene graph via the data channel. ### Claim 28 (depends on 25) The wireless device of claim 25, further comprising: means for sending an offer to send or to receive the scene graph to the other of the plurality of participant computing devices as part of session initiation protocol (SIP) setup for the immersive three-dimensional group session. ### Claim 29 (depends on 25) The wireless device of claim 25, further comprising: means for sending an offer to send or to receive the scene graph to the other of the plurality of participant computing devices, wherein the offer indicates the own graphical output node. ### Claim 30 (depends on 25) The wireless device of claim 25, wherein the immersive three-dimensional group session is a Web Real-Time Communications (WebRTC) session.