# US 2024/0205036 A1 · Shared augmented reality experience in video chat

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0205036 A1
- Document ID: US20240205036A1
- Status: Published application
- Kind: A1
- Application number: 18/589,175
- Filed: 2024-02-27
- Published: 2024-06-20
- Earliest claimed: 2022-04-25
- Assignees: Snap Inc.
- Applicants: Snap Inc.
- Inventors: Nathan Richard Banks; Nathan Kenneth Boyd; Amanda Durham; Alex Edelsburg; Maxim Maximov Lazarvo; Ryan Thomas
- Web page: https://patentgrep.com/patent/US20240205036A1

## CPC

- H04L 12/1818: {Conference organisation arrangements, e.g. handling schedules, setting up parameters needed by nodes to attend a conference, booking network resources, notifying involved parties}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04L 12/1822: {Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission}
- H04L 51/52: for supporting social networking services
- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- H04N 7/15: Conference systems
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- G06T 2219/024: Multi-user, collaborative environment
- H04N 2007/145: {Handheld terminals}
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Methods and systems are disclosed for performing operations for providing a shared augmented reality experience in a video chat. A video chat can be established between a plurality of client devices. During the video chat, videos of users associated with the client devices can be displayed. During the video chat, a request from a first client device to activate a first AR experience can be received, and in response, and body parts of users depicted in the videos are modified to include one or more AR elements associated with the first AR experience.

## Claims

### Claim 1

A method comprising:
adding a third device to an established video call between first and second devices;
determining that a first augmented reality (AR) experience is inactive by the third device; and
dividing a graphical user interface into a plurality of cells comprising first and second cells in response to determining that the first AR experience is inactive by the third device, a first cell of the plurality of cells representing a user of the third device and a second cell of the plurality of cells comprising first and second videos associated with the first and second devices, the first and second videos modified based on the first AR experience.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
generating, for concurrent display in the graphical user interface, the first video received from the first device and the second video received from the second device;
receiving a request from the first device to activate the first AR experience during the video call; and
in response to receiving the request to activate the first AR experience:
cropping a first body part of a first user depicted in the first video and a second body part of a second user depicted in the second video;
modifying the first and second body parts using one or more AR elements associated with the first AR experience; and
adding the first and second body parts that have been modified using the one or more AR elements to a common background representing a virtual environment of the first AR experience.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
displaying on the first device a list of options each associated with a different AR experience; and
detecting input from the first device that selects a first option from the list of options associated with the first AR experience.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
obtaining a first list of identifiers of shared AR experiences available on the first device;
obtaining a second list of identifiers of shared AR experiences available on the second device;
comparing the first and second lists of identifiers of the shared experiences to identify a set of shared AR experiences that is available in both the first list of identifiers and the second list of identifiers; and
displaying the set of shared AR experiences that is available in both the first list of identifiers and the second list of identifiers.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
determining that the first AR experience is currently unavailable to the second device; and
in response to determining that the first AR experience is currently unavailable to the second device, causing an option to be displayed on the second device to download the first AR experience or receive temporary access to the first AR experience and automatically authorize activation of the first AR experience, wherein the second video is modified in response to receiving a selection of the option from the second device.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
detecting a first face of a first user in the first video and a second face of a second user in the second video;
applying a first AR element associated with the first AR experience to the first face; and
applying a second AR element associated with the first AR experience to the second face.

### Claim 7 (depends on 6)

The method of claim 6, wherein the first AR element comprises a first virtual character, and wherein the second AR element comprises a second virtual character.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
generating a display of a virtual environment, wherein the first and second virtual characters are presented in the virtual environment together with names of the respective first and second users.

### Claim 9 (depends on 8)

The method of claim 8, wherein the virtual environment comprises a fishbowl, and wherein the first and second virtual characters comprise different types of fish.

### Claim 10 (depends on 6)

The method of claim 6, further comprising:
generating a display of a virtual environment comprising a virtual couch, wherein the first and second AR elements are presented in the virtual environment on the virtual couch together with names of the respective first and second users.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
after adding first and second body parts that have been modified using one or more AR elements of the first AR experience to a common background representing a virtual environment of the first AR experience, preventing the second device from switching the first AR experience to a second AR experience until a threshold period of time elapses.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
identifying a plurality of different AR elements associated with the first AR experience; and
computing a count representing how many video feeds are currently being presented concurrently.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
retrieving a set of different AR elements from the plurality of different AR elements based on the computed count, wherein each of the different AR elements in the set of different AR elements is used to modify first and second body parts.

### Claim 14 (depends on 1)

The method of claim 1, further comprising:
receiving a third video from the third device;
displaying the third video in the first cell;
determining that a second AR experience has been activated on the third device; and
generating a display of the third video in the first cell modified based on the second AR experience together with a display of the first and second videos modified based on the first AR experience in the second cell.

### Claim 15 (depends on 1)

The method of claim 1, further comprising:
receiving a request from the third device to activate the first AR experience; and
in response to receiving the request from the third device to activate the first AR experience, modifying a representation of the user of the third device based on the first AR experience.

### Claim 16 (depends on 15)

The method of claim 15, further comprising removing the first cell from the graphical user interface in response to modifying the representation of the user of the third device.

### Claim 17

A system comprising:
at least one processor; and
a memory component having instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
adding a third device to an established video call between first and second devices;
determining that a first augmented reality (AR) experience is inactive by the third device; and
dividing a graphical user interface into a plurality of cells comprising first and second cells in response to determining that the first AR experience is inactive by the third device, a first cell of the plurality of cells representing a user of the third device and a second cell of the plurality of cells comprising first and second videos associated with the first and second devices, the first and second videos modified based on the first AR experience.

### Claim 18

A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
adding a third device to an established video call between first and second devices;
determining that a first augmented reality (AR) experience is inactive by the third device; and
dividing a graphical user interface into a plurality of cells comprising first and second cells in response to determining that the first AR experience is inactive by the third device, a first cell of the plurality of cells representing a user of the third device and a second cell of the plurality of cells comprising first and second videos associated with the first and second devices, the first and second videos modified based on the first AR experience.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium of claim 18, the operations comprising:
generating, for concurrent display in the graphical user interface, the first video received from the first device and the second video received from the second device;
receiving a request from the first device to activate the first AR experience during the video call; and
in response to receiving the request to activate the first AR experience:
cropping a first body part of a first user depicted in the first video and a second body part of a second user depicted in the second video;
modifying the first and second body parts using one or more AR elements associated with the first AR experience; and
adding the first and second body parts that have been modified using the one or more AR elements to a common background representing a virtual environment of the first AR experience.

### Claim 20 (depends on 18)

The non-transitory computer-readable storage medium of claim 18, the operations comprising:
displaying on the first device a list of options each associated with a different AR experience; and
detecting input from the first device that selects a first option from the list of options associated with the first AR experience.
