# US 12,182,903 B2 · Augmented reality based communication between multiple users > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,182,903 B2 - Document ID: US12182903B2 - Status: Granted - Kind: B2 - Application number: 17/206,991 - Filed: 2021-03-19 - Published: 2021-09-30 - Granted: 2024-12-31 - Earliest claimed: 2020-03-25 - Assignees: Snap Inc. - Applicants: Snap Inc. - Inventors: Brian Anthony Smith; Yu Jiang Tham; Rajan Vaish; Hemant Surale - Web page: https://patentgrep.com/patent/US12182903B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06T 11/00: Two-dimensional [2D] image generation - G02B 27/0172: {characterised by optical features} - G06F 3/1454: {involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay} - H04L 65/1059: End-user terminal functionalities specially adapted for real-time communication - H04L 65/1063: Application servers providing network services (systems providing special services to telephonic subscribers H04M3/42) - H04L 65/1069: Session establishment or de-establishment - H04L 65/1089: by adding media; by removing media - H04L 65/61: for supporting one-way streaming services, e.g. Internet radio - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - G06F 3/147: using display panels - G09G 2340/12: Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels - G09G 2354/00: Aspects of interface with display user - H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, method, and user interface to facilitate augmented reality based communication between multiple users over a network. Input data is received from a first device that is indicative of a selection by a first user of virtual content item to apply to a real-world environment that is visible to a second user via a second device. The virtual content item may comprise one or more media objects. Based on receiving the input data, the second device is caused to present the one or more media objects overlaid on the real-world environment that is visible to the second user via the second device. ## Claims ### Claim 1 A method comprising: receiving, from a first device worn by a first user, session configuration data for a virtual interaction session with the first user, the session configuration data including configuration parameters of the virtual interaction session specified by the first user, the configuration parameters indicating that a second user and a third user are permitted by the first user to join the virtual interaction session with the first user, the configuration parameters including: a session duration that defines a first time limit for the virtual interaction session, and a micro-chat duration that defines a second time limit for one or more real-time communication links with the first user during the virtual interaction session; based on the configuration parameters indicating that the second user is permitted to join the virtual interaction session, causing display, by a second device of the second user, of a first interactive element that enables the second user to establish a first real-time communication link with the first user during the virtual interaction session; based on the configuration parameters indicating that the third user is permitted to join the virtual interaction session, causing display, by a third device of the third user, of a second interactive element until expiration of the first time limit defined by the session duration, the second interactive element enabling the third user to establish a second real-time communication link with the first user during the virtual interaction session; based on receiving, from the second device of the second user, an indication of an interaction with the first interactive element by the second user, establishing the first real-time communication link between the first user and the second user by performing operations including causing display, by the second device, of a live camera feed generated by a camera of the first device, the live camera feed comprising images of a real-world environment that is visible to the first user via a transparent display of the first device worn by the first user; based on the display of the live camera feed generated by the camera of the first device being displayed by the second device of the second user, causing display, by the transparent display of the first device, of an indicator that the second user is viewing the live camera feed generated by the camera of the first device; verifying, based on the session configuration data, that the second user is permitted by the first user to apply virtual content items to the real-world environment that is visible to the first user; based on the verifying, causing, by one or more processors, the transparent display of the first device to present a virtual content item selected by the second user overlaid on the real-world environment that is visible to the first user via the transparent display of the first device; augmenting the display of the live camera feed by the second device to include the virtual content item overlaid on the real-world environment that is visible to the first user via the transparent display of the first device; based on detecting expiration of the second time limit defined by the micro-chat duration, terminating the first real-time communication link between the first user and the second user; upon termination of the first real-time communication link between the first user and the second user, providing a third interactive element to the second device to trigger transmission of a request for a third real-time communication link between the first user and the second user; and establishing the third real-time communication link between the first user and the second user in response to receiving the request prior to expiration of the session duration; and detecting expiration of the first time limit defined by the session duration, the third user being unable to establish the second real-time communication link with the first user based on the expiration of the first time limit defined by the session duration. ### Claim 2 (depends on 1) The method of claim 1, further comprising: causing display, at the second device, of a user interface that includes a set of selectable virtual content items to apply to the real-world environment that is visible to the first user, wherein the second user selects the virtual content item from the set of selectable virtual content items. ### Claim 3 (depends on 1) The method of claim 1, wherein the indicator comprises an identifier of the second user. ### Claim 4 (depends on 1) The method of claim 1, further comprising enabling an audio communication link between the first and second devices while the live camera feed is displayed by the second device. ### Claim 5 (depends on 1) The method of claim 1, wherein: the indicator is a first indicator; and the method further comprising causing display, by the first device and the second device, of a second indicator of time remaining for the display of the live camera feed by the second device based on the micro-chat duration. ### Claim 6 A system comprising: a memory that stores instructions; and one or more processors configured by the instructions to perform operations comprising: receiving, from a first device worn by a first user, session configuration data for a virtual interaction session with the first user, the session configuration data including configuration parameters of the virtual interaction session specified by the first user, the configuration parameters indicating that a second user and a third user are permitted by the first user to join the virtual interaction session with the first user, the configuration parameters including: a session duration that defines a time limit for the virtual interaction session, and a micro-chat duration that defines a second time limit for one or more real-time communication links with the first user during the virtual interaction session; based on the configuration parameters indicating that the second user is permitted to join the virtual interaction session, causing display, by a second device of the second user, of a first interactive element that enables the second user to establish a first real-time communication link with the first user during the virtual interaction session; based on the configuration parameters indicating that the third user is permitted to join the virtual interaction session, causing display, by a third device of the third user, of a second interactive element until expiration of the first time limit defined by the session duration, the second interactive element enabling the third user to establish a second real-time communication link with the first user during the virtual interaction session; based on receiving, from the second device of the second user, an indication of an interaction with the first interactive element by the second user, establishing the first real-time communication link between the first user and the second user by performing operations including causing display, by the second device, of a live camera feed generated by a camera of the first device, the live camera feed comprising images of a real-world environment that is visible to the first user via a transparent display of the first device worn by the first user; based on the display of the live camera feed generated by the camera of the first device being displayed by the second device of the second user, causing display, by the transparent display of the first device, of an indicator that the second user is viewing the live camera feed generated by the camera of the first device; verifying, based on the session configuration data, that the second user is permitted by the first user to apply virtual content items to the real-world environment that is visible to the first user; based on the verifying, causing the transparent display of the first device to present a virtual content item selected by the second user overlaid on the real-world environment that is visible to the first user via the transparent display of the first device; augmenting the display of the live camera feed by the second device to include the virtual content item overlaid on the real-world environment that is visible to the first user via the transparent display of the first device; based on detecting expiration of the second time limit defined by the micro-chat duration, terminating the real-time communication link between the first user and the second user; upon termination of the first real-time communication link between the first user and the second user, providing a third interactive element to the second device to trigger transmission of a request for a third real-time communication link between the first user and the second user; establishing the third real-time communication link between the first user and the second user in response to receiving the request prior to expiration of the session duration; and detecting expiration of the first time limit defined by the session duration, the third user being unable to establish the second real-time communication link with the first user based on the expiration of the first time limit defined by the session duration. ### Claim 7 (depends on 6) The system of claim 6, wherein the operations further comprise: causing display, at the second device, of a user interface that includes a set of selectable virtual content items to apply to the real-world environment that is visible to the first user, wherein the second user selects the virtual content item from the set of selectable virtual content items. ### Claim 8 (depends on 6) The system of claim 6, wherein the indicator comprises an identifier of the second user. ### Claim 9 (depends on 6) The system of claim 6, wherein the operations further comprise enabling an audio communication link between the first and second devices while the live camera feed is displayed by the second device. ### Claim 10 (depends on 6) The system of claim 6, wherein: the indicator is a first indicator; and the operations further comprise causing display, by the first device and the second device, of a second indicator of time remaining for the display of the live camera feed by the second device based on the micro-chat duration. ### Claim 11 A machine-readable medium storing instructions that, when executed by a computer system, cause the computer system to perform operations comprising: receiving, from a first device worn by a first user, session configuration data for a virtual interaction session with the first user, the session configuration data including configuration parameters of the virtual interaction session specified by the first user, the configuration parameters indicating that a second user and a third user permitted by the first user to join the virtual interaction session with the first user, the configuration parameters including: a session duration that defines a time limit for the virtual interaction session, and a micro-chat duration that defines a second time limit for one or more real-time communication links with the first user during the virtual interaction session; based on the configuration parameters indicating that the second user is permitted to join the virtual interaction session, causing display, by a second device of the second user, of a first interactive element that enables the second user to establish a first real-time communication link with the first user during the virtual interaction session; based on the configuration parameters indicating that the third user is permitted to join the virtual interaction session, causing display, by a third device of the third user, of a second interactive element until expiration of the first time limit defined by the session duration, the second interactive element enabling the third user to establish a second real-time communication link with the first user during the virtual interaction session; based on receiving, from the second device of the second user, an indication of an interaction with the first interactive element by the second user, establishing the first real-time communication link between the first user and the second user by performing operations including causing display, by the second device, of a live camera feed generated by a camera of the first device, the live camera feed comprising images of a real-world environment that is visible to the first user via a transparent display of the first device worn by the first user; based on the display of the live camera feed generated by the camera of the first device being displayed by the second device of the second user, causing display, by the transparent display of the first device, of an indicator that the second user is viewing the live camera feed generated by the camera of the first device; verifying, based on the session configuration data, that the second user is permitted by the first user to apply virtual content items to the real-world environment that is visible to the first user; based on the verifying, causing the transparent display of the first device to present a virtual content item selected by the second user overlaid on the real-world environment that is visible to the first user via the transparent display of the first device; augmenting the display of the live camera feed by the second device to include the virtual content item overlaid on the real-world environment that is visible to the first user via the transparent display of the first device; based on detecting expiration of the second time limit defined by the micro-chat duration, terminating the real-time communication link between the first user and the second user; and upon termination of the first real-time communication link between the first user and the second user, providing a third interactive element to the second device to trigger transmission of a request for a third real-time communication link between the first user and the second user; and establishing the third real-time communication link between the first user and the second user in response to receiving the request prior to expiration of the session duration; and detecting expiration of the first time limit defined by the session duration, the third user being unable to establish the second real-time communication link with the first user based on the expiration of the first time limit defined by the session duration. ### Claim 12 (depends on 11) The machine-readable medium of claim 11, wherein the operations further comprise: causing display, at the second device, of a user interface that includes a set of selectable virtual content items to apply to the real-world environment that is visible to the first user, wherein the second user selects the virtual content item from the set of selectable virtual content items. ### Claim 13 (depends on 11) The machine-readable medium of claim 11, wherein the operations further comprise enabling an audio communication link between the first and second devices while the live camera feed is displayed by the second device. ### Claim 14 (depends on 11) The machine-readable medium of claim 11, wherein: the indicator is a first indicator; and the operations further comprise causing display, by the first device and the second device, of a second indicator of time remaining for the display of the live camera feed by the second device based on the micro-chat duration.