# US 12,113,948 B1 · Systems and methods of managing spatial groups in multi-user communication sessions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,113,948 B1
- Document ID: US12113948B1
- Status: Granted
- Kind: B1
- Application number: 18/421,827
- Filed: 2024-01-24
- Granted: 2024-10-08
- Earliest claimed: 2023-06-04
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Connor A. Smith; Willem Mattelaer; Joseph P. Cerra; Kevin Lee
- Web page: https://patentgrep.com/patent/US12113948B1

## CPC

- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06T 15/20: Perspective computation
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- G06T 2219/024: Multi-user, collaborative environment
- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Some examples of the disclosure are directed to systems and methods for managing locations of users in a spatial group within a communication session based on the display of shared content in a three-dimensional environment. In some examples, a first electronic device and a second electronic device are in communication within a communication session. In some examples, the first electronic device displays a three-dimensional environment including an avatar corresponding to a user of the second electronic device. In some examples, in response to detecting an input corresponding to a request to display shared content in the three-dimensional environment, if the shared content is a first type of content, the first electronic positions the avatar a first distance away from the viewpoint, and if the shared content is a second type of content, the first electronic device positions the avatar a second distance away from the viewpoint.

## Claims

### Claim 1

A method comprising:
at a first electronic device in communication with a display, one or more input devices, a second electronic device and a third electronic device:
while in a communication session with the second electronic device and the third electronic device, displaying, via the display, a computer-generated environment including a three-dimensional representation corresponding to a user of the second electronic device at a first location and a three-dimensional representation corresponding to a user of the third electronic device at a second location, different from the first location, in the computer-generated environment relative to a viewpoint of the first electronic device;
while displaying the computer-generated environment including the three-dimensional representation corresponding to the user of the second electronic device and the three-dimensional representation corresponding to the user of the third electronic device, receiving, via the one or more input devices, a first input corresponding to a request to display content in the computer-generated environment; and
in response to receiving the first input, in accordance with a determination that the content corresponds to shared content:
displaying, via the display, a first object corresponding to the shared content in the computer-generated environment; and
displaying the three-dimensional representation corresponding to the user of the second electronic device at a first updated location and the three-dimensional representation corresponding to the user of the third electronic device at a second updated location, different from the first updated location, in the computer-generated environment relative to the viewpoint, including:
moving the three-dimensional representation of the user of the second electronic device to the first updated location and moving the three-dimensional representation of the user of the third electronic device to the second updated location in a respective direction that is selected based on a location of the first object.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first object is a shared application window associated with an application operating on the first electronic device.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first updated location and the second updated location are determined relative to a reference line in the computer-generated environment.

### Claim 4 (depends on 3)

The method of claim 3, wherein:
before receiving the first input, the user of the first electronic device, the user of the second electronic device, and the user of the third electronic device are arranged within a spatial group having a center point; and
the reference line extends between the location of the first object in the computer-generated environment and the center point of the spatial group.

### Claim 5 (depends on 4)

The method of claim 4, wherein the center point is determined based on a calculated average of the viewpoint of the first electronic device, the first location, and the second location.

### Claim 6 (depends on 3)

The method of claim 3, wherein:
the respective direction of movement of the three-dimensional representation corresponding to the user of the second electronic device is clockwise relative to the reference line in the computer-generated environment; and
the respective direction of movement of the three-dimensional representation corresponding to the user of the third electronic device is counterclockwise relative to the reference line in the computer-generated environment.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
before receiving the first input, detecting an indication of a change in state of the second electronic device; and
in response to detecting the indication:
replacing display of the three-dimensional representation corresponding to the user of the second electronic device with a two-dimensional representation of the user of the second electronic device; and
displaying the three-dimensional representation corresponding to the user of the third electronic device at a third updated location relative to the viewpoint, including:
moving the three-dimensional representation of the user of the third electronic device to the third updated location in the respective direction that is selected based on a location of the two-dimensional representation of the user of the second electronic device.

### Claim 8 (depends on 1)

The method of claim 1, wherein the first electronic device, the second electronic device, and the third electronic device include a head-mounted display, respectively.

### Claim 9 (depends on 1)

The method of claim 1, wherein the three-dimensional representation of the user of the second electronic device and the three-dimensional representation of the user of the third electronic device are moved to the first updated location and the second updated location, respectively, in the respective direction with an animation of the movement.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
in response to receiving the first input, in accordance with a determination that the content corresponds to private content:
displaying, via the display, a second object corresponding to the private content in the computer-generated environment; and
maintaining display of the three-dimensional representation corresponding to the user of the second electronic device at the first location and maintaining display of the three-dimensional representation corresponding to the user of the third electronic device at the second location in the computer-generated environment.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
while displaying the computer-generated environment including the three-dimensional representation corresponding to the user of the second electronic device, the three-dimensional representation corresponding to the user of the third electronic device, and the second object, receiving, via the one or more input devices, a second input corresponding to a request to share the private content with the user of the second electronic device and the user of the third electronic device; and
in response to receiving the second input:
redisplaying the second object as a shared object in the computer-generated environment; and
displaying the three-dimensional representation corresponding to the user of the second electronic device at a third updated location and the three-dimensional representation corresponding to the user of the third electronic device at a fourth updated location, different from the third updated location, in the computer-generated environment relative to the viewpoint, including:
moving the three-dimensional representation of the user of the second electronic device to the third updated location and moving the three-dimensional representation of the user of the third electronic device to the fourth updated location in the respective direction that is selected based on a location of the second object.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
while displaying the computer-generated environment including the three-dimensional representation corresponding to the user of the second electronic device and the three-dimensional representation corresponding to the user of the third electronic device, detecting an indication of a request to display shared content in the computer-generated environment; and
in response to detecting the indication:
displaying, via the display, a second object corresponding to the shared content in the computer-generated environment; and
updating the viewpoint of the first electronic device in the computer-generated environment relative to a location of the second object.

### Claim 13 (depends on 1)

The method of claim 1, wherein the viewpoint of the first electronic device, the first location, and the second location are arranged according to a spatial group in the computer-generated environment, the method further comprising:
while displaying the computer-generated environment including the three-dimensional representation corresponding to the user of the second electronic device and the three-dimensional representation corresponding to the user of the third electronic device, detecting an indication that the user of the second electronic device is no longer in the communication session; and
in response to detecting the indication:
ceasing display of the three-dimensional representation corresponding to the user of the second electronic device in the computer-generated environment; and
maintaining display of the three-dimensional representation corresponding to the user of the third electronic device at the second location in the computer-generated environment.

### Claim 14 (depends on 13)

The method of claim 13, further comprising:
while displaying the computer-generated environment including the three-dimensional representation corresponding to the user of the third electronic device, detecting an indication that a user of a fourth electronic device has joined the communication session; and
in response to detecting the indication:
displaying, via the display, a three-dimensional representation corresponding to the user of the fourth electronic device at the first location in the computer-generated environment.

### Claim 15 (depends on 13)

The method of claim 13, further comprising:
while displaying the computer-generated environment including the three-dimensional representation corresponding to the user of the third electronic device, receiving, via the one or more input devices, a second input corresponding to a request to display shared content in the computer-generated environment; and
in response to receiving the second input:
displaying, via the display, a respective object corresponding to the shared content in the computer-generated environment; and
displaying the three-dimensional representation corresponding to the user of the third electronic device at a third location, different from the first location and the second location, in the computer-generated environment.

### Claim 16 (depends on 15)

The method of claim 15, wherein:
prior receiving the second input, the second location at which the three-dimensional representation corresponding to the user of the third electronic device is displayed is a first distance from the viewpoint of the first electronic device; and
in response to receiving the second input, the third location at which the three-dimensional representation corresponding to the user of the third electronic device is displayed is a second distance, smaller than the first distance, from the viewpoint.

### Claim 17 (depends on 13)

The method of claim 13, wherein the computer-generated environment further includes a respective object corresponding to shared content, the method further comprising:
while displaying the computer-generated environment including the three-dimensional representation corresponding to the user of the third electronic device and the respective object, receiving, via the one or more input devices, a second input corresponding to a request to cease display of the shared content in the computer-generated environment; and
in response to receiving the second input:
ceasing display of the respective object in the computer-generated environment; and
displaying the three-dimensional representation corresponding to the user of the third electronic device at a third location, different from the first location and the second location, in the computer-generated environment.

### Claim 18 (depends on 1)

The method of claim 1, wherein the viewpoint of the first electronic device, the first location, and the second location are arranged according to a spatial group in the computer-generated environment, the method further comprising:
while displaying the computer-generated environment including the three-dimensional representation corresponding to the user of the second electronic device and the three-dimensional representation corresponding to the user of the third electronic device, detecting an indication that a user of a fourth electronic device has joined the communication session; and
in response to detecting the indication:
displaying, via the display, a three-dimensional representation corresponding to the user of the fourth electronic device at a third location in the computer-generated environment;
moving the three-dimensional representation corresponding to the user of the second electronic device to a fourth location, different from the first location, in the computer-generated environment; and
moving the three-dimensional representation corresponding to the user of the third electronic device to a fifth location, different from the second location, in the computer-generated environment.

### Claim 19

An electronic device comprising:
one or more processors;
memory; and
one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing a method comprising:
while in a communication session with a second electronic device and a third electronic device, displaying, via a display, a computer-generated environment including a three-dimensional representation corresponding to a user of the second electronic device at a first location and a three-dimensional representation corresponding to a user of the third electronic device at a second location, different from the first location, in the computer-generated environment relative to a viewpoint of the electronic device;
while displaying the computer-generated environment including the three-dimensional representation corresponding to the user of the second electronic device and the three-dimensional representation corresponding to the user of the third electronic device, receiving, via one or more input devices, a first input corresponding to a request to display content in the computer-generated environment; and
in response to receiving the first input, in accordance with a determination that the content corresponds to shared content:
displaying, via the display, a first object corresponding to the shared content in the computer-generated environment; and
displaying the three-dimensional representation corresponding to the user of the second electronic device at a first updated location and the three-dimensional representation corresponding to the user of the third electronic device at a second updated location, different from the first updated location, in the computer-generated environment relative to the viewpoint, including:
moving the three-dimensional representation of the user of the second electronic device to the first updated location and moving the three-dimensional representation of the user of the third electronic device to the second updated location in a respective direction that is selected based on a location of the first object.

### Claim 20

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the electronic device to perform a method comprising:
while in a communication session with a second electronic device and a third electronic device, displaying, via a display, a computer-generated environment including a three-dimensional representation corresponding to a user of the second electronic device at a first location and a three-dimensional representation corresponding to a user of the third electronic device at a second location, different from the first location, in the computer-generated environment relative to a viewpoint of the electronic device;
while displaying the computer-generated environment including the three-dimensional representation corresponding to the user of the second electronic device and the three-dimensional representation corresponding to the user of the third electronic device, receiving, via one or more input devices, a first input corresponding to a request to display content in the computer-generated environment; and
in response to receiving the first input, in accordance with a determination that the content corresponds to shared content:
displaying, via the display, a first object corresponding to the shared content in the computer-generated environment; and
displaying the three-dimensional representation corresponding to the user of the second electronic device at a first updated location and the three-dimensional representation corresponding to the user of the third electronic device at a second updated location, different from the first updated location, in the computer-generated environment relative to the viewpoint, including:
moving the three-dimensional representation of the user of the second electronic device to the first updated location and moving the three-dimensional representation of the user of the third electronic device to the second updated location in a respective direction that is selected based on a location of the first object.