# US 2021/0240262 A1 · Systems and methods for assisting virtual gestures based on viewing frustum

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0240262 A1
- Document ID: US20210240262A1
- Status: Published application
- Kind: A1
- Application number: 17/238,565
- Filed: 2021-04-23
- Published: 2021-08-05
- Earliest claimed: 2016-06-28
- Assignees: Rec Room Inc.
- Applicants: Rec Room Inc.
- Inventors: Nicholas Fajt; Cameron Brown; Dan Kroymann; Omer Bilal Orhan; Johnathan Bevis; Joshua Wehrly
- Web page: https://patentgrep.com/patent/US20210240262A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- A63F 13/21: characterised by their sensors, purposes or types
- A63F 13/211: using inertial sensors, e.g. accelerometers or gyroscopes
- A63F 13/212: using sensors worn by the player, e.g. for measuring heart beat or leg activity
- A63F 13/25: Output arrangements for video game devices
- A63F 13/35: Details of game servers
- A63F 13/5255: according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
- A63F 13/573: using trajectories of game objects, e.g. of a golf ball according to the point of impact
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 17/10: Complex mathematical operations {(function generation by table look-up G06F1/03; evaluation of elementary functions by calculation G06F7/544)}
- A63F 2300/8082: Virtual reality
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR

## Abstract

An endpoint system including one or more computing devices presents an object in a virtual environment (e.g., a shared virtual environment); receives gaze input corresponding to a gaze of a user of the endpoint system; calculates a gaze vector based on the gaze input; receives motion input corresponding to an action of the user; determines a path adjustment (e.g., by changing motion parameters such as trajectory and velocity) for the object based at least in part on the gaze vector and the motion input; and simulates motion of the object within the virtual environment based at least in part on the path adjustment. The object may be presented as being thrown by an avatar, with a flight path based on the path adjustment. The gaze vector may be based on head orientation information, eye tracking information, or some combination of these or other gaze information.

## Claims

### Claim 1

A system comprising:
an endpoint system including one or more computing devices programmed to:
present at least a portion of an avatar in a virtual environment, wherein the avatar is associated with the endpoint system, and wherein the portion of the avatar is associated with a gaze-activation region;
calculate a gaze vector of a user of the endpoint system;
determine whether the gaze vector intersects or is within a threshold distance of the gaze-activation region; and
present a user interface within the virtual environment if the gaze vector intersects or is within the threshold distance of the gaze-activation region.

### Claim 2 (depends on 1)

The system of claim 1, wherein the gaze-activation region is movable in the virtual environment.

### Claim 3 (depends on 1)

The system of claim 1, wherein the gaze-activation region is movable along with the portion of the avatar in response to information received from a handheld controller.

### Claim 4 (depends on 1)

The system of claim 1, wherein presenting the user interface comprises:
presenting the user interface in an initial position based on the position of the portion of the avatar in the virtual environment; and
updating the initial position of the user interface, such that the user interface moves when the portion of the avatar moves.

### Claim 5 (depends on 1)

The system of claim 1, wherein presenting the user interface comprises:
presenting the user interface in an initial position based on the position of the portion of the avatar in the virtual environment;
receiving user input; and
detaching the user interface from the portion of the avatar in response to the user input.

### Claim 6 (depends on 1)

The system of claim 1, wherein the gaze-activation region is represented in the virtual environment as an item worn by the avatar.

### Claim 7 (depends on 1)

The system of claim 1, wherein the user interface comprises a plurality of user-selectable elements, wherein the one or more computing devices are further programmed to transmit notifications in response to activation of the user-selectable elements, and wherein the notifications are transmitted via a communication relay server.

### Claim 8 (depends on 1)

The system of claim 1, wherein the virtual environment is a shared virtual environment.

### Claim 9 (depends on 1)

The system of claim 1, wherein the endpoint system comprises:
an endpoint computing device;
a head-mounted display device communicatively coupled to the endpoint computing device; and
at least one handheld controller device communicatively coupled to the endpoint computing device.

### Claim 10

A computer-implemented method comprising:
by an endpoint system:
presenting at least a portion of an avatar in a virtual environment, wherein the avatar is associated with the endpoint system, and wherein the portion of the avatar is associated with a gaze-activation region;
calculating a gaze vector of a user of the endpoint system;
determining whether the gaze vector intersects or is within a threshold distance of the gaze-activation region; and
presenting a user interface within the virtual environment if the gaze vector intersects or is within a threshold distance of the gaze-activation region.

### Claim 11 (depends on 10)

The method of claim 10, wherein the gaze-activation region is movable in the virtual environment.

### Claim 12 (depends on 10)

The method of claim 10, wherein the gaze-activation region is movable along with the portion of the avatar in response to information received from a handheld controller.

### Claim 13 (depends on 10)

The method of claim 10, wherein presenting the user interface comprises:
presenting the user interface in an initial position based on the position of the portion of the avatar in the virtual environment; and
updating the initial position of the user interface, such that the user interface moves when the portion of the avatar moves.

### Claim 14 (depends on 10)

The method of claim 10, wherein presenting the user interface comprises:
presenting the user interface in an initial position based on the position of the portion of the avatar in the virtual environment;
receiving user input; and
detaching the user interface from the portion of the avatar in response to the user input.

### Claim 15 (depends on 10)

The method of claim 10, wherein the gaze-activation region is represented in the virtual environment as an item worn by the avatar.

### Claim 16 (depends on 10)

The method of claim 10, wherein the user interface comprises a plurality of user-selectable elements, the method further comprising transmitting notifications via a communication relay server in response to activation of the user-selectable elements.

### Claim 17 (depends on 10)

The method of claim 10, wherein the virtual environment is a shared virtual environment.

### Claim 18

A non-transitory computer-readable medium having computer-executable instructions that, in response to execution by at least one processor of at least one computing device, cause the at least one computing device to perform a method comprising:
presenting at least a portion of an avatar in a virtual environment, wherein the avatar is associated with the endpoint system, and wherein the portion of the avatar is associated with a gaze-activation region;
calculating a gaze vector of a user of the endpoint system;
determining whether the gaze vector intersects or is within a threshold distance of the gaze-activation region; and
presenting a user interface within the virtual environment if the gaze vector intersects or is within a threshold distance of the gaze-activation region.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the gaze-activation region is movable in the virtual environment.

### Claim 20 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the gaze-activation region is movable along with the portion of the avatar in response to information received from a handheld controller.

### Claim 21 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein presenting the user interface comprises:
presenting the user interface in an initial position based on the position of the portion of the avatar in the virtual environment; and
updating the initial position of the user interface, such that the user interface moves when the portion of the avatar moves.

### Claim 22 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein presenting the user interface comprises:
presenting the user interface in an initial position based on the position of the portion of the avatar in the virtual environment;
receiving user input; and
detaching the user interface from the portion of the avatar in response to the user input.

### Claim 23 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the gaze-activation region is represented in the virtual environment as an item worn by the avatar.

### Claim 24 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the user interface comprises a plurality of user-selectable elements, the method further comprising transmitting notifications via a communication relay server in response to activation of the user-selectable elements.

### Claim 25 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the virtual environment is a shared virtual environment.
