# US 2019/0356758 A1 · Methods for visualizing and interacting with a three dimensional object in a collaborative augmented reality environment and apparatuses thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0356758 A1
- Document ID: US20190356758A1
- Status: Published application
- Kind: A1
- Application number: 16/415,446
- Filed: 2019-05-17
- Published: 2019-11-21
- Earliest claimed: 2018-05-17
- Assignees: Kaon Interactive, Inc.
- Applicants: Kaon Interactive
- Inventors: Gavin Finn; Joshua Smith; Anatoly Dedkov
- Web page: https://patentgrep.com/patent/US20190356758A1

## CPC

- H04L 67/131: Protocols for games, networked simulations or virtual reality
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06F 3/1454: {involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay}
- G06F 3/147: using display panels
- G06T 19/003: {Navigation within 3D models or images}
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2219/024: Multi-user, collaborative environment
- G06T 2219/2016: Rotation, translation, scaling
- G09G 2340/0492: Change of orientation of the displayed image, e.g. upside-down, mirrored
- G09G 2356/00: Detection of the display position w.r.t. other display screens
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00)

## Abstract

Methods, non-transitory computer readable media, and collaborative computing apparatus that establish a collaborative session for visualizing and interacting with a three-dimensional object in a collaborative augmented reality environment between two or more of a plurality of computing devices. Position and orientation information of each of the two or more of the plurality of computing devices is obtained. An interaction instruction with respect to the three-dimensional object from a first of the two or more of the plurality of computing devices is received. Instructions for adjusting visualization of the three-dimensional object on each of the other of the two or more of the plurality of computing devices are determined and provided based on the received interaction instruction and the obtained position and orientation information of each of the two or more of the plurality of computing devices.

## Claims

### Claim 1

A method comprising:
establishing, by an collaborative computing apparatus, a collaborative session for visualizing and interacting with a three-dimensional object in a collaborative augmented reality environment between two or more of a plurality of computing devices;
obtaining, by the collaborative computing apparatus, position and orientation information of each of the two or more of the plurality of computing devices;
receiving, by the collaborative computing apparatus, an interaction instruction with respect to the three-dimensional object from a first of the two or more of the plurality of computing devices; and
determining and providing, by the collaborative computing apparatus, one or more instructions to adjust visualization of the three-dimensional object on each of the other of the two or more of the plurality of computing devices based on the received interaction instruction and the obtained position and orientation information of each of the two or more of the plurality of computing devices.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving, by the collaborative computing apparatus, a join request from each of the two or more of the plurality of computing devices, wherein the join request has an embedded master status indicator tag.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
determining, by the collaborative computing apparatus, when one of the two or more of the plurality of computing devices is a master device based on the embedded master status indicator tag; and
registering, by the collaborative computing apparatus, the one of the two or more of the plurality of computing devices as the master device when the determination indicates that the one of the two or more of the plurality of computing devices is the master device.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
receiving, by the collaborative computing apparatus, another interaction instruction with respect to the three-dimensional object from a second of the two or more of the plurality of computing devices; and
adjusting, by the collaborative computing apparatus, visualization of the three-dimensional object on each of the other of the two or more of the plurality of computing devices based on the received interaction instruction and the obtained position and orientation information of each of the two or more of the plurality of computing devices.

### Claim 5 (depends on 1)

The method of claim 1, wherein the received interaction instruction comprises rotation of the 3D object model, manipulation of the 3D object model or drawing over the 3D object model.

### Claim 6 (depends on 1)

The method of claim 1, wherein the obtaining position and orientation information of each of the two or more of the plurality of computing devices further comprises:
tracking, by the collaborative computing apparatus, six degrees of freedom of each of the two or more of the plurality of computing devices; and
determining, by the collaborative computing apparatus, position and orientation information associated with each of the two or more of the plurality of computing devices based on the tracking.

### Claim 7

A collaborative computing device, comprising memory comprising programmed instructions stored thereon and one or more processors configured to be capable of executing the stored programmed instructions to:
establish a collaborative session for visualizing and interacting with a three-dimensional object in a collaborative augmented reality environment between two or more of a plurality of computing devices;
obtain position and orientation information of each of the two or more of the plurality of computing devices;
receive an interaction instruction with respect to the three-dimensional object from a first of the two or more of the plurality of computing devices; and
determine and provide one or more instructions to adjust visualization of the three-dimensional object on each of the other of the two or more of the plurality of computing devices based on the received interaction instruction and the obtained position and orientation information of each of the two or more of the plurality of computing devices.

### Claim 8 (depends on 7)

The computing device of claim 7, wherein the one or more processors are further configured to be capable of executing the stored programmed instructions to:
receive a join request from each of the two or more of the plurality of computing devices, wherein the join request has an embedded master status indicator tag.

### Claim 9 (depends on 8)

The computing device of claim 8, wherein the one or more processors are further configured to be capable of executing the stored programmed instructions to:
determine when one of the two or more of the plurality of computing devices is a master device based on the embedded master status indicator tag; and
register the one of the two or more of the plurality of computing devices as the master device when the determination indicates that the one of the two or more of the plurality of computing devices is the master device.

### Claim 10 (depends on 7)

The computing device of claim 7, wherein the one or more processors are further configured to be capable of executing the stored programmed instructions to:
receive another interaction instruction with respect to the three-dimensional object from a second of the two or more of the plurality of computing devices; and
adjust visualization of the three-dimensional object on each of the other of the two or more of the plurality of computing devices based on the received interaction instruction and the obtained position and orientation information of each of the two or more of the plurality of computing devices.

### Claim 11 (depends on 7)

The computing device of claim 7, wherein the received interaction instruction comprises rotation of the 3D object model, manipulation of the 3D object model or drawing over the 3D object model.

### Claim 12 (depends on 7)

The computing device of claim 7, wherein the one or more processors are further configured to be capable of executing the stored programmed instructions to:
track six degrees of freedom of each of the two or more of the plurality of computing devices; and
determine position and orientation information associated with each of the two or more of the plurality of computing devices based on the tracking.

### Claim 13

A non-transitory computer readable medium having stored thereon instructions comprising executable code which when executed by one or more processors, causes the one or more processors to:
establish a collaborative session for visualizing and interacting with a three-dimensional object in a collaborative augmented reality environment between two or more of a plurality of computing devices;
obtain position and orientation information of each of the two or more of the plurality of computing devices;
receive an interaction instruction with respect to the three-dimensional object from a first of the two or more of the plurality of computing devices; and
determine and provide one or more instructions to adjust visualization of the three-dimensional object on each of the other of the two or more of the plurality of computing devices based on the received interaction instruction and the obtained position and orientation information of each of the two or more of the plurality of computing devices.

### Claim 14 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the executable code when executed by the one or more processors further causes the one or more processors to:
receive a join request from each of the two or more of the plurality of computing devices, wherein the join request has an embedded master status indicator tag.

### Claim 15 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the executable code when executed by the one or more processors further causes the one or more processors to:
determine when one of the two or more of the plurality of computing devices is a master device based on the embedded master status indicator tag; and
register the one of the two or more of the plurality of computing devices as the master device when the determination indicates that the one of the two or more of the plurality of computing devices is the master device.

### Claim 16 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the executable code when executed by the one or more processors further causes the one or more processors to:
receive another interaction instruction with respect to the three-dimensional object from a second of the two or more of the plurality of computing devices; and
adjust visualization of the three-dimensional object on each of the other of the two or more of the plurality of computing devices based on the received interaction instruction and the obtained position and orientation information of each of the two or more of the plurality of computing devices.

### Claim 17 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the received interaction instruction comprises rotation of the 3D object model, manipulation of the 3D object model or drawing over the 3D object model.

### Claim 18 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein the executable code when executed by the one or more processors further causes the one or more processors to:
track six degrees of freedom of each of the two or more of the plurality of computing devices; and
determine position and orientation information associated with each of the two or more of the plurality of computing devices based on the tracking.
