# US 12,112,011 B2 · System and method of application-based three-dimensional refinement in multi-user communication sessions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,112,011 B2
- Document ID: US12112011B2
- Status: Granted
- Kind: B2
- Application number: 18/465,098
- Filed: 2023-09-11
- Published: 2024-03-21
- Granted: 2024-10-08
- Earliest claimed: 2022-09-16
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Connor A. Smith; Christopher D. McKenzie; Nathan Gitter
- Web page: https://patentgrep.com/patent/US12112011B2

## CPC

- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Some examples of the disclosure are directed to methods for application-based spatial refinement in a multi-user communication session including a first electronic device and a second electronic device. While the first electronic device is presenting a three-dimensional environment, the first electronic device receives an input corresponding to a request to move a shared object in the three-dimensional environment. In accordance with a determination that the shared object is an object of a first type, the first electronic device moves the shared object and an avatar of a user in the three-dimensional environment in accordance with the input. In accordance with a determination that the shared object is an object of a second type, different from the first type, and the input is a first type of input, the first electronic device moves the shared object in the three-dimensional environment in accordance with the input, without moving the avatar.

## Claims

### Claim 1

A method comprising:
at a first electronic device in communication with a display, one or more input devices, and a second electronic device:
while in a communication session with the second electronic device, presenting, via the display, a computer-generated environment including an avatar corresponding to a user of the second electronic device and a first shared object;
while presenting the computer-generated environment including the avatar corresponding to the user of the second electronic device and the first shared object, receiving, via the one or more input devices, a first input corresponding to a request to move the first shared object in a first manner in the computer-generated environment; and
in response to receiving the first input:
in accordance with a determination that the first shared object is an object of a first type, moving the avatar and the first shared object in the computer-generated environment in the first manner in accordance with the first input; and
in accordance with a determination that the first shared object is an object of a second type that is different from the first type and the first input is a first type of input, moving the first shared object in the computer-generated environment in the first manner in accordance with the first input without moving the avatar.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first type of input corresponds to one or more of:
a change in distance between a viewpoint of a user of the first electronic device and the first shared object; and
vertical movement of the first shared object in the computer-generated environment relative to the viewpoint of the user.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
in response to receiving the first input:
in accordance with a determination that the first shared object is an object of the second type and the first input is a second type of input, different from the first type of input, moving the avatar and the first shared object in the computer-generated environment in the first manner in accordance with the first input, wherein the second type of input corresponds to radial lateral movement relative to a viewpoint of a user of the first electronic device.

### Claim 4 (depends on 1)

The method of claim 1, wherein, before receiving the first input, the computer-generated environment further includes a first unshared object, the method further comprising:
while presenting the computer-generated environment including the avatar corresponding to the user of the second electronic device and the first shared object, receiving, via the one or more input devices, a second input corresponding to a request to move the first unshared object in the computer-generated environment; and
in response to receiving the second input:
moving the first unshared object in the computer-generated environment in accordance with the second input without moving the avatar and the first shared object.

### Claim 5 (depends on 1)

The method of claim 1, wherein the object of the first type corresponds to an object that has a horizontal orientation relative to a viewpoint of a user of the first electronic device, and wherein the object of the second type corresponds to an object that has a vertical orientation relative to a viewpoint of a user of the first electronic device.

### Claim 6 (depends on 1)

The method of claim 1, wherein the first input includes:
a pinch gesture provided by a hand of a user of the first electronic device; and
movement of the hand of the user while holding the pinch gesture with the hand.

### Claim 7 (depends on 1)

The method of claim 1, wherein moving the first shared object in the first manner corresponds to:
moving the first shared object toward a viewpoint of the user of the first electronic device; or
moving the first shared object away from the viewpoint of the user of the first electronic device.

### Claim 8 (depends on 1)

The method of claim 1, wherein, before receiving the first input, the first shared object has a first orientation in the computer-generated environment, the method further comprising:
while presenting the computer-generated environment including the avatar corresponding to the user of the second electronic device and the first shared object, receiving, via the one or more input devices, a second input corresponding to a request to move the first shared object laterally in the computer-generated environment relative to a viewpoint of a user of the first electronic device; and
in response to receiving the second input:
in accordance with the determination that the first shared object is an object of the first type or that the first shared object is an object of the second type, moving the avatar and the first shared object laterally in the computer-generated environment relative to the viewpoint of the user in accordance with the second input.

### Claim 9

A first electronic device comprising:
one or more processors;
memory; and
one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing a method comprising:
while in a communication session with a second electronic device, presenting, via a display, a computer-generated environment including an avatar corresponding to a user of the second electronic device and a first shared object;
while presenting the computer-generated environment including the avatar corresponding to the user of the second electronic device and the first shared object, receiving, via one or more input devices, a first input corresponding to a request to move the first shared object in a first manner in the computer-generated environment; and
in response to receiving the first input:
in accordance with a determination that the first shared object is an object of a first type, moving the avatar and the first shared object in the computer-generated environment in the first manner in accordance with the first input; and
in accordance with a determination that the first shared object is an object of a second type that is different from the first type and the first input is a first type of input, moving the first shared object in the computer-generated environment in the first manner in accordance with the first input without moving the avatar.

### Claim 10 (depends on 9)

The first electronic device of claim 9, wherein the first type of input corresponds to one or more of:
a change in distance between a viewpoint of a user of the first electronic device and the first shared object; and
vertical movement of the first shared object in the computer-generated environment relative to the viewpoint of the user.

### Claim 11 (depends on 9)

The first electronic device of claim 9, wherein the method further comprises:
in response to receiving the first input:
in accordance with a determination that the first shared object is an object of the second type and the first input is a second type of input, different from the first type of input, moving the avatar and the first shared object in the computer-generated environment in the first manner in accordance with the first input, wherein the second type of input corresponds to radial lateral movement relative to a viewpoint of a user of the first electronic device.

### Claim 12 (depends on 9)

The first electronic device of claim 9, wherein, before receiving the first input, the computer-generated environment further includes a first unshared object, the method further comprising:
while presenting the computer-generated environment including the avatar corresponding to the user of the second electronic device and the first shared object, receiving, via the one or more input devices, a second input corresponding to a request to move the first unshared object in the computer-generated environment; and
in response to receiving the second input:
moving the first unshared object in the computer-generated environment in accordance with the second input without moving the avatar and the first shared object.

### Claim 13 (depends on 9)

The first electronic device of claim 9, wherein the object of the first type corresponds to an object that has a horizontal orientation relative to a viewpoint of a user of the first electronic device, and wherein the object of the second type corresponds to an object that has a vertical orientation relative to a viewpoint of a user of the first electronic device.

### Claim 14 (depends on 9)

The first electronic device of claim 9, wherein the first input includes:
a pinch gesture provided by a hand of a user of the first electronic device; and
movement of the hand of the user while holding the pinch gesture with the hand.

### Claim 15 (depends on 9)

The first electronic device of claim 9, wherein moving the first shared object in the first manner corresponds to:
moving the first shared object toward a viewpoint of the user of the first electronic device; or
moving the first shared object away from the viewpoint of the user of the first electronic device.

### Claim 16 (depends on 9)

The first electronic device of claim 9, wherein, before receiving the first input, the first shared object has a first orientation in the computer-generated environment, the method further comprising:
while presenting the computer-generated environment including the avatar corresponding to the user of the second electronic device and the first shared object, receiving, via the one or more input devices, a second input corresponding to a request to move the first shared object laterally in the computer-generated environment relative to a viewpoint of a user of the first electronic device; and
in response to receiving the second input:
in accordance with the determination that the first shared object is an object of the first type or that the first shared object is an object of the second type, moving the avatar and the first shared object laterally in the computer-generated environment relative to the viewpoint of the user in accordance with the second input.

### Claim 17

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of a first electronic device, cause the first electronic device to perform a method comprising:
while in a communication session with a second electronic device, presenting, via a display, a computer-generated environment including an avatar corresponding to a user of the second electronic device and a first shared object;
while presenting the computer-generated environment including the avatar corresponding to the user of the second electronic device and the first shared object, receiving, via one or more input devices, a first input corresponding to a request to move the first shared object in a first manner in the computer-generated environment; and
in response to receiving the first input:
in accordance with a determination that the first shared object is an object of a first type, moving the avatar and the first shared object in the computer-generated environment in the first manner in accordance with the first input; and
in accordance with a determination that the first shared object is an object of a second type that is different from the first type and the first input is a first type of input, moving the first shared object in the computer-generated environment in the first manner in accordance with the first input without moving the avatar.

### Claim 18 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the first type of input corresponds to one or more of:
a change in distance between a viewpoint of a user of the first electronic device and the first shared object; and
vertical movement of the first shared object in the computer-generated environment relative to the viewpoint of the user.

### Claim 19 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the method further comprises:
in response to receiving the first input:
in accordance with a determination that the first shared object is an object of the second type and the first input is a second type of input, different from the first type of input, moving the avatar and the first shared object in the computer-generated environment in the first manner in accordance with the first input, wherein the second type of input corresponds to radial lateral movement relative to a viewpoint of a user of the first electronic device.

### Claim 20 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein, before receiving the first input, the computer-generated environment further includes a first unshared object, the method further comprising:
while presenting the computer-generated environment including the avatar corresponding to the user of the second electronic device and the first shared object, receiving, via the one or more input devices, a second input corresponding to a request to move the first unshared object in the computer-generated environment; and
in response to receiving the second input:
moving the first unshared object in the computer-generated environment in accordance with the second input without moving the avatar and the first shared object.

### Claim 21 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the object of the first type corresponds to an object that has a horizontal orientation relative to a viewpoint of a user of the first electronic device, and wherein the object of the second type corresponds to an object that has a vertical orientation relative to a viewpoint of a user of the first electronic device.

### Claim 22 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein the first input includes:
a pinch gesture provided by a hand of a user of the first electronic device; and
movement of the hand of the user while holding the pinch gesture with the hand.

### Claim 23 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein moving the first shared object in the first manner corresponds to:
moving the first shared object toward a viewpoint of the user of the first electronic device; or
moving the first shared object away from the viewpoint of the user of the first electronic device.

### Claim 24 (depends on 17)

The non-transitory computer readable storage medium of claim 17, wherein, before receiving the first input, the first shared object has a first orientation in the computer-generated environment, the method further comprising:
while presenting the computer-generated environment including the avatar corresponding to the user of the second electronic device and the first shared object, receiving, via the one or more input devices, a second input corresponding to a request to move the first shared object laterally in the computer-generated environment relative to a viewpoint of a user of the first electronic device; and
in response to receiving the second input:
in accordance with the determination that the first shared object is an object of the first type or that the first shared object is an object of the second type, moving the avatar and the first shared object laterally in the computer-generated environment relative to the viewpoint of the user in accordance with the second input.