# US 11,909,791 B2 · Synchronization in a multiuser experience > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,909,791 B2 - Document ID: US11909791B2 - Status: Granted - Kind: B2 - Application number: 18/189,403 - Filed: 2023-03-24 - Published: 2023-09-28 - Granted: 2024-02-20 - Earliest claimed: 2020-09-24 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Luis R. Deliz Centeno; Joseph P. Cerra - Web page: https://patentgrep.com/patent/US11909791B2 ## CPC - H04L 51/10: Multimedia information - H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - H04L 12/1827: {Network arrangements for conference optimisation or adaptation} - H04L 51/18: Commands or executable codes - G06T 2219/024: Multi-user, collaborative environment - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Facilitating collaboration in a multiuser communication session includes detecting initiation of an interaction between a user of a first device and a virtual object in a multiuser communication session, where the first device and a second device are active in the multiuser communication session, and where state information is synchronized among the first device and second device in the multiuser communication session. In accordance with detecting the initiation of an interaction between the user and the virtual object, synchronization of state information is suspended by the first device, where the state information is related to the interaction between the user and the virtual object. A representation of the multiuser communication session at the first device and a representation of the multiuser communication session at the second device are inconsistent in accordance with the suspended synchronization of the state information. ## Claims ### Claim 1 A method for facilitating collaboration in a multiuser communication session, comprising: providing, at a first device, a representation of a multiuser communication session in which the first device and a second device are active, wherein the representation of the multiuser communication session comprises a virtual object; synchronizing the representation of the multiuser communication session at the first device with a representation of the multiuser communication session at the second device by providing, to the second device, user state information associated with a pose of a user of the first device and object state information associated with a state of the virtual object at the first device, wherein the virtual object is presented consistently among the first device and the second device in accordance with the synchronization; detecting, at the first device, initiation of an interaction between the user of the first device and the virtual object, wherein the interaction causes a change in presentation of the virtual object at the first device; and in accordance with detecting the initiation of the interaction between the user and the virtual object, suspending the synchronization by ceasing to provide, to the second device, the user state information and the object state information, wherein the presentation of the virtual object at the first device and at the second device are inconsistent in accordance with the suspended synchronization. ### Claim 2 (depends on 1) The method of claim 1, wherein the object state information is associated with a presentation state of the virtual object. ### Claim 3 (depends on 2) The method of claim 2, wherein the presentation state of the virtual object comprises a configuration of a presentation of the virtual object within the representation of the multiuser communication session at the first device. ### Claim 4 (depends on 1) The method of claim 1, further comprising: in accordance with detecting a change in location of the user, transmitting updated user state information associated with an updated location to the second device. ### Claim 5 (depends on 1) The method of claim 1, further comprising: in accordance with detecting completion of the interaction between the user and the virtual object, obtaining an updated presentation state of the representation of the multiuser communication session for the second device; and presenting an updated representation of the multiuser communication session in accordance with the updated presentation state for the second device. ### Claim 6 (depends on 5) The method of claim 5, wherein the updated presentation state is received from the second device prior to the completion of the interaction. ### Claim 7 (depends on 5) The method of claim 5, wherein the representation of the multiuser communication session is further presented in accordance with the interaction between the user and the virtual object. ### Claim 8 (depends on 5) The method of claim 5, further comprising: further in accordance with detecting completion of the interaction between the user and the virtual object, obtaining an updated presentation state for a third device active in the multiuser communication session. ### Claim 9 (depends on 5) The method of claim 5, wherein presenting the updated representation comprises: interpolating one or more intermediary frames between a current representation of the multiuser communication session at the first device and an updated representation of the multiuser communication session for the second device. ### Claim 10 (depends on 5) The method of claim 5, wherein the completion of the interaction is detected based on determining an elapsed time since the user interacted with the virtual object satisfies a predetermined threshold. ### Claim 11 (depends on 1) The method of claim 1, further comprising: in accordance with detecting completion of the interaction between the user and the virtual object and determining that the first device is authorized to globally update a state of the representation of the multiuser communication session, transmitting a current state of the representation of the multiuser communication session for the first device to the second device. ### Claim 12 (depends on 1) The method of claim 1, further comprising: in accordance with detecting the initiation of the interaction between the user and the virtual object, transmitting an indication of the suspended synchronization of the state information to the second device. ### Claim 13 (depends on 1) The method of claim 1, wherein a representation of the user and the virtual object in the multiuser communication session at the second device is inconsistent with the interaction of the user and the virtual object in accordance with the suspended synchronization of the state information. ### Claim 14 (depends on 1) The method of claim 1, further comprising: in accordance with a detection of a commit action by the first device, providing additional state information for the first device, wherein the representation of the multiuser communication session at the first device is configured to be synchronized with the representation of the multiuser communication session at the second device in accordance with the additional state information, and wherein the virtual object is presented consistently among the first device and the second device in accordance with the additional state information. ### Claim 15 A non-transitory computer-readable medium comprising computer-readable code executable by one or more processors to: provide, at a first device, a representation of a multiuser communication session in which the first device and a second device are active, wherein the representation of the multiuser communication session comprises a virtual object; synchronize the representation of the multiuser communication session at the first device with a representation of the multiuser communication session at the second device by providing, to the second device, user state information associated with a pose of a user of the first device and object state information associated with a state of the virtual object at the first device, wherein the virtual object is presented consistently among the first device and the second device in accordance with the synchronization; detect, at the first device, initiation of an interaction between the user of the first device and the virtual object, wherein the interaction causes a change in presentation of the virtual object at the first device; and in accordance with detecting the initiation of the interaction between the user and the virtual object, suspend the synchronization by ceasing to provide, to the second device, the user state information and the object state information, wherein the presentation of the virtual object at the first device and at the second device are inconsistent in accordance with the suspended synchronization. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the object state information is associated with a presentation state of the virtual object. ### Claim 17 (depends on 16) The non-transitory computer-readable medium of claim 16, wherein the presentation state of the virtual object comprises a configuration of the presentation of the virtual object within a representation of the multiuser communication session at the first device. ### Claim 18 (depends on 15) The non-transitory computer-readable medium of claim 15, further comprising computer-readable code to: in accordance with detecting a change in location of the user, transmit updated user state information associated with an updated location to the second device. ### Claim 19 A system comprising: one or more processors; and one or more non-transitory computer-readable media comprising computer-readable code executable by the one or more processors to: provide, at a first device, a representation of a multiuser communication session in which the first device and a second device are active, wherein the representation of the multiuser communication session comprises a virtual object; synchronize the representation of the multiuser communication session at the first device with a representation of the multiuser communication session at the second device by providing, to the second device, user state information associated with a pose of a user of the first device and object state information associated with a state of the virtual object at the first device, wherein the virtual object is presented consistently among the first device and the second device in accordance with the synchronization; detect, at the first device, initiation of an interaction between the user of the first device and the virtual object, wherein the interaction causes a change in presentation of the virtual object at the first device; and in accordance with detecting the initiation of the interaction between the user and the virtual object, suspend the synchronization by ceasing to provide, to the second device, the user state information and the object state information, wherein the presentation of the virtual object at the first device and at the second device are inconsistent in accordance with the suspended synchronization. ### Claim 20 (depends on 19) The system of claim 19, further comprising computer-readable code to: in accordance with detecting completion of the interaction between the user and the virtual object, obtain an updated presentation state of the representation of the multiuser communication session for the second device; and present an updated representation of the multiuser communication session in accordance with the updated presentation state for the second device.