US 10,242,501 B1Granted

Multi-user virtual and augmented reality tracking systems

AssigneeWorldViz, Inc.

InventorsMatthias Pusch, Andrew C. Beall

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems and methods are described for virtual and augmented reality. A position of a first user and a position of a second user in a physical space are received, wherein the positions are derived from detected infrared light from a plurality of cameras. An image corresponding to the first user is rendered at a first virtual position in a display device associated with the second user, wherein the first virtual position is determined based at least in part on a position of the first user in the physical space and on a determined viewpoint of the second user. An image corresponding to the second user is rendered at a second virtual position in a display device associated with the first user, wherein the second virtual position is determined based at least in part on a position of the second user in the physical space and on a determined viewpoint of the first user.

Claims21

Dependent claims

A cluster system, comprising:

a computing device configured as a master computing device;
a plurality of motion tracking cameras, wherein the plurality of motion tracking cameras is configured to reside in a physical space coincident with a first user and a second user, the plurality of motion tracking cameras configured to detect infrared light associated with the first user and infrared light associated with the second user, and to communicate position data derived from detected infrared light associated with the first user and detected infrared light associated with the second user to the master computing device;
non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to:
provide a user interface via which the first user and/or an administrator may designate at least one element of a scene as a gaze direction condition for which an indication is to be rendered;
access configuration information comprising information indicating what types of operations are privileged and what types of operations are non-privileged;
receive, at the master computing device from one or more of the plurality of motion tracking cameras, position data corresponding to a position of the first user in the physical space and position data corresponding to a position of the second user in the physical space;
cause, by the master computing device, an image corresponding to the first user to be rendered at a first virtual position in a display device associated with the second user, wherein the first virtual position is determined at least in part on the position data corresponding to the position of the first user in the physical space and on a determined viewpoint of the second user;
cause, by the master computing device, an image corresponding to the second user to be rendered at a second virtual position in a display device associated with the first user, wherein the second virtual position is based at least in part on the position data corresponding to the position of the second user in the physical space and on a determined viewpoint of the first user;
receive at the master computing device, eye-tracking position data associated with the second user;
and
determine, based at least in part on the received eye-tracking position data associated with the second user and on the determined viewpoint of the second user, a gaze direction of the second user;
and
cause the indication to be rendered in the display device associated with the first user at least partly in response to determining that the gaze direction of the second user satisfies the gaze direction condition designated via the user interface.

10 more elements

A cluster system, comprising:

a computing device configured as a master computing device;
a plurality of motion tracking cameras, wherein the plurality of motion tracking cameras is configured to reside in a physical space coincident with a first user and a second user, the plurality of motion tracking cameras configured to detect infrared light associated with the first user and infrared light associated with the second user, and to communicate position data derived from detected infrared light associated with the first user and detected infrared light associated with the second user to the master computing device;
non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to:
access configuration information comprising information indicating what types of operations are privileged and what types of operations are non-privileged;
receive, at the master computing device from one or more of the plurality of motion tracking cameras, position data corresponding to a position of the first user in the physical space and position data corresponding to a position of the second user in the physical space;
cause, by the master computing device, an image corresponding to the first user to be rendered at a first virtual position in a display device associated with the second user, wherein the first virtual position is determined at least in part on the position data corresponding to the position of the first user in the physical space and on a determined viewpoint of the second user;
cause, by the master computing device, an image corresponding to the second user to be rendered at a second virtual position in a display device associated with the first user, wherein the second virtual position is based at least in part on the position data corresponding to the position of the second user in the physical space and on a determined viewpoint of the first user:
receive at the master computing device, eye-tracking position data associated with the second user;
determine, based at least in part on the received eye-tracking position data associated with the second user and on the determined viewpoint of the second user, a gaze direction of the second user;
and
cause an indication to be rendered in the display device associated with the first user at least partly in response to determining that the gaze direction of the second user is not toward a first element of a scene.

8 more elements

A cluster system, comprising:

a computing device configured as a master computing device;
a plurality of motion tracking cameras, wherein the plurality of motion tracking cameras is configured to reside in a physical space coincident with a first user and a second user, the plurality of motion tracking cameras configured to detect infrared light associated with the first user and infrared light associated with the second user, and to communicate position data derived from detected infrared light associated with the first user and detected infrared light associated with the second user to the master computing device;
non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to:
access configuration information comprising information indicating what types of operations are privileged and what types of operations are non-privileged;
receive, at the master computing device from one or more of the plurality of motion tracking cameras, position data corresponding to a position of the first user in the physical space and position data corresponding to a position of the second user in the physical space;
cause, by the master computing device, an image corresponding to the first user to be rendered at a first virtual position in a display device associated with the second user, wherein the first virtual position is determined at least in part on the position data corresponding to the position of the first user in the physical space and on a determined viewpoint of the second user;
cause, by the master computing device, an image corresponding to the second user to be rendered at a second virtual position in a display device associated with the first user, wherein the second virtual position is based at least in part on the position data corresponding to the position of the second user in the physical space and on a determined viewpoint of the first user;
receive at the master computing device, eye-tracking position data associated with the second user;
determine, based at least in part on the received eye-tracking position data associated with the second user and on the determined viewpoint of the second user, a gaze direction of the second user;
and
at least partly in response to a determination that the gaze of the second user is directed at the first user, cause a depiction of the second user's eyes to be rendered in the display device associated with the first user, wherein the depiction of the second user's eyes depicts the second user's eyes as being aligned with the first user's eyes.

8 more elements

A cluster system, comprising:

a computing device configured as a master computing device;
a plurality of motion tracking cameras, wherein the plurality of motion tracking cameras is configured to reside in a physical space coincident with a first user and a second user, the plurality of motion tracking cameras configured to detect infrared light associated with the first user and infrared light associated with the second user, and to communicate position data derived from detected infrared light associated with the first user and detected infrared light associated with the second user to the master computing device;
non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to:
access configuration information comprising information indicating what types of operations are privileged and what types of operations are non-privileged;
receive, at the master computing device from one or more of the plurality of motion tracking cameras, position data corresponding to a position of the first user in the physical space and position data corresponding to a position of the second user in the physical space;
cause, by the master computing device, an image corresponding to the first user to be rendered at a first virtual position in a display device associated with the second user, wherein the first virtual position is determined at least in part on the position data corresponding to the position of the first user in the physical space and on a determined viewpoint of the second user;
cause, by the master computing device, an image corresponding to the second user to be rendered at a second virtual position in a display device associated with the first user, wherein the second virtual position is based at least in part on the position data corresponding to the position of the second user in the physical space and on a determined viewpoint of the first user;
receive at the master computing device, eye-tracking position data associated with the second user;
determine, based at least in part on the received eye-tracking position data associated with the second user and on the determined viewpoint of the second user, a gaze direction of the second user;
and
wherein the second user comprises a plurality of users, and wherein the cluster system is further configured to cause an indication to be rendered in the display device associated with the first user at least partly in response to determining that the gaze direction of at least one user of the plurality of users satisfies one or more conditions, and wherein the rendered indication includes an identifier associated with the at least one user whose gaze is causing the one or more conditions to be satisfied.

8 more elements

A cluster system, comprising:

a computing device configured as a master computing device;
a plurality of motion tracking cameras, wherein the plurality of motion tracking cameras is configured to reside in a physical space coincident with a first user and a second user, the plurality of motion tracking cameras configured to detect infrared light associated with the first user and infrared light associated with the second user, and to communicate position data derived from detected infrared light associated with the first user and detected infrared light associated with the second user to the master computing device;
non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to:
access configuration information comprising information indicating what types of operations are privileged and what types of operations are non-privileged;
receive, at the master computing device from one or more of the plurality of motion tracking cameras, position data corresponding to a position of the first user in the physical space and position data corresponding to a position of the second user in the physical space;
cause, by the master computing device, an image corresponding to the first user to be rendered at a first virtual position in a display device associated with the second user, wherein the first virtual position is determined at least in part on the position data corresponding to the position of the first user in the physical space and on a determined viewpoint of the second user;
cause, by the master computing device, an image corresponding to the second user to be rendered at a second virtual position in a display device associated with the first user, wherein the second virtual position is based at least in part on the position data corresponding to the position of the second user in the physical space and on a determined viewpoint of the first user;
receive at the master computing device, eye-tracking position data associated with the second user;
determine, based at least in part on the received eye-tracking position data associated with the second user and on the determined viewpoint of the second user, a gaze direction of the second user;
and
wherein the second user comprises a plurality of users, and wherein the cluster system is further configured to cause an indication to be rendered in the display device associated with the first user at least partly in response to determining that at least a threshold number of users in the plurality of users have a determined gaze direction that satisfies one or more conditions, the threshold number greater than one.

8 more elements

A cluster system, comprising:

a computing device;
non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to:
receive, from a plurality of motion tracking cameras, position data for a first user and a second user in one or more physical spaces, the position data for the first user and the second user derived from detected infrared light associated respectively with the first user and the second user;
access configuration information comprising information indicating what types of operations are privileged and what types of operations are non-privileged;
cause an image corresponding to the first user to be rendered at a first virtual position in a display device associated with the second user, wherein the first virtual position is determined based at least in part on received position data for the first user in a first physical space and on a determined viewpoint of the second user;
cause an image corresponding to the second user to be rendered at a second virtual position in a display device associated with the first user, wherein the second virtual position is determined based at least in part on received position data for the second user in a second physical space and on a determined viewpoint of the first user;
receive image data from a camera associated with the display device associated with the second user;
determine, based at least in part on the received image data from the camera associated with the display device associated with the second user and on the determined viewpoint of the second user, a gaze direction of the second user;
and
cause an indication to be rendered in the display device associated with the first user at least partly in response to determining that the gaze direction of the second user is not toward a first element of a scene.

7 more elements

A cluster system, comprising:

a computing device;
non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to:
provide a user interface via which a first user and/or an administrator may designate at least one element of a scene as a gaze direction condition for which an indication is to be rendered;
receive, from a plurality of motion tracking cameras, position data for the first user and a second user in one or more physical spaces, the position data for the first user and the second user derived from detected infrared light associated respectively with the first user and the second user;
access configuration information comprising information indicating what types of operations are privileged and what types of operations are non-privileged;
cause an image corresponding to the first user to be rendered at a first virtual position in a display device associated with the second user, wherein the first virtual position is determined based at least in part on received position data for the first user in a first physical space and on a determined viewpoint of the second user;
cause an image corresponding to the second user to be rendered at a second virtual position in a display device associated with the first user, wherein the second virtual position is determined based at least in part on received position data for the second user in a second physical space and on a determined viewpoint of the first user;
receive image data from a camera associated with the display device associated with the second user;
determine, based at least in part on the received image data from the camera associated with the display device associated with the second user and on the determined viewpoint of the second user, a gaze direction of the second user;
and
cause the indication to be rendered in the display device associated with the first user at least partly in response to determining that the gaze direction of the second user satisfies the gaze direction condition designated via the user interface.

8 more elements

A cluster system, comprising:

a computing device;
non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to:
receive, from a plurality of motion tracking cameras, position data for a first user and a second user in one or more physical spaces, the position data for the first user and the second user derived from detected infrared light associated respectively with the first user and the second user;
access configuration information comprising information indicating what types of operations are privileged and what types of operations are non-privileged;
cause an image corresponding to the first user to be rendered at a first virtual position in a display device associated with the second user, wherein the first virtual position is determined based at least in part on received position data for the first user in a first physical space and on a determined viewpoint of the second user;
cause an image corresponding to the second user to be rendered at a second virtual position in a display device associated with the first user, wherein the second virtual position is determined based at least in part on received position data for the second user in a second physical space and on a determined viewpoint of the first user;
receive image data from a camera associated with the display device associated with the second user;
determine, based at least in part on the received image data from the camera associated with the display device associated with the second user and on the determined viewpoint of the second user, a gaze direction of the second user;
and
at least partly in response to a determination that the gaze of the second user is directed at the first user, cause a depiction of the second user's eyes to be rendered in the display device associated with the first user, wherein the depiction of the second user's eyes depicts the second user's eyes as being aligned with the first user's eyes.

7 more elements

A cluster system, comprising:

a computing device;
non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to:
receive, from a plurality of motion tracking cameras, position data for a first user and a second user in one or more physical spaces, the position data for the first user and the second user derived from detected infrared light associated respectively with the first user and the second user;
access configuration information comprising information indicating what types of operations are privileged and what types of operations are non-privileged;
cause an image corresponding to the first user to be rendered at a first virtual position in a display device associated with the second user, wherein the first virtual position is determined based at least in part on received position data for the first user in a first physical space and on a determined viewpoint of the second user;
cause an image corresponding to the second user to be rendered at a second virtual position in a display device associated with the first user, wherein the second virtual position is determined based at least in part on received position data for the second user in a second physical space and on a determined viewpoint of the first user;
receive image data from a camera associated with the display device associated with the second user;
determine, based at least in part on the received image data from the camera associated with the display device associated with the second user and on the determined viewpoint of the second user, a gaze direction of the second user;
and
wherein the second user comprises a plurality of users, and wherein the cluster system is further configured to cause an indication to be rendered in the display device associated with the first user at least partly in response to determining that the gaze direction of at least one user of the plurality of users satisfies one or more conditions, and wherein the rendered indication includes an identifier associated with the at least one user whose gaze is causing the one or more conditions to be satisfied.

7 more elements

A cluster system, comprising:

a computing device;
non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to:
receive, from a plurality of motion tracking cameras, position data for a first user and a second user in one or more physical spaces, the position data for the first user and the second user derived from detected infrared light associated respectively with the first user and the second user;
access configuration information comprising information indicating what types of operations are privileged and what types of operations are non-privileged;
cause an image corresponding to the first user to be rendered at a first virtual position in a display device associated with the second user, wherein the first virtual position is determined based at least in part on received position data for the first user in a first physical space and on a determined viewpoint of the second user;
cause an image corresponding to the second user to be rendered at a second virtual position in a display device associated with the first user, wherein the second virtual position is determined based at least in part on received position data for the second user in a second physical space and on a determined viewpoint of the first user;
receive image data from a camera associated with the display device associated with the second user;
determine, based at least in part on the received image data from the camera associated with the display device associated with the second user and on the determined viewpoint of the second user, a gaze direction of the second user;
and
wherein the second user comprises a plurality of users, and wherein the cluster system is further configured to cause an indication to be rendered in the display device associated with the first user at least partly in response to determining that at least a threshold number of users in the plurality of users have a determined gaze direction that satisfies one or more conditions, the threshold number greater than one.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/496,984Filed 25 Apr 2017 · us provisional application
Priority: 3 May 2016
Granted: 26 Mar 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification27 codes

G06T 19/006
Manipulating three-dimensional [3D] models or images for computer graphics: mixed reality
G02B 27/0172
Head mounted: characterised by optical features
G06F 3/011
Input arrangements or combined input and output arrangements for interaction between user and computer: arrangements for interaction with the…
G06F 3/013
Arrangements for interaction with the human body, e.g. for user immersion in virtual reality: eye tracking input arrangements