# US 2021/0240985 A1 · 360-degree video for large scale navigation with 3d interactable models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0240985 A1
- Document ID: US20210240985A1
- Status: Published application
- Kind: A1
- Application number: 16/779,444
- Filed: 2020-01-31
- Published: 2021-08-05
- Earliest claimed: 2020-01-31
- Assignees: Honeywell International Inc.
- Applicants: Honeywell International Inc.
- Inventors: Mark Phillips; Ziad Kaakani; Jason Urso
- Web page: https://patentgrep.com/patent/US20210240985A1

## CPC

- G06T 19/003: {Navigation within 3D models or images}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06V 20/20: in augmented reality scenes
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06V 20/647: {by matching two-dimensional images to three-dimensional objects}
- G06F 2203/012: Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2219/2016: Rotation, translation, scaling
- G06V 2201/06: Recognition of objects for industrial automation
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method and system for authoring a three-dimensional (3D) presentation can involve dividing a scene into a grid of equals-sized segments or non-equally sized segments, wherein one or more 360-degree cameras record the scene from a center of each segment, and a size of the each segment is proportional to a complexity of an environment captured in the scene. One or more objects can be created within the scene for a user interaction with a 3D presentation of the scene in the context of a video stream. A logical center of the object(s) can be used as a center point for recording a 360-degree video by the one or more 360-degree cameras for use as a low fidelity background video when the user interacts with the 3D presentation of the scene. The 3D presentation can be based on a 3D model.

## Claims

### Claim 1

A method of authoring a three-dimensional (3D) presentation, comprising:
dividing a scene into a grid of segments, wherein at least one 360-degree camera records the scene from a center of each segment among the segments, wherein a size of the each segment is proportional to a complexity of an environment captured in the scene;
creating at least one object within the scene for a user interaction with a 3D presentation of the scene in a context of a video stream; and
using a logical center of the at least one object as a center point for recording a 360-degree video by the at least one 360-degree camera for use as a low fidelity background video when the user interacts with the 3D presentation of the scene, wherein the 3D presentation is based on a 3D model.

### Claim 2 (depends on 1)

The method of claim 1 wherein the scene comprises a large-scale scene.

### Claim 3 (depends on 1)

The method of claim 1 wherein the at least one 360-degree video comprises at least one visual cue.

### Claim 4 (depends on 1)

The method of claim 1 further comprising navigating between a plurality of hotspots in the 3D presentation using a teleporting method that translates a position of a user from a current hotspot among the plurality of hotspots and provides a motion blur as the video stream is transitioned to a new hotspot among the plurality of hotspots, which becomes a new position of the user.

### Claim 5 (depends on 4)

The method of claim 4 wherein each hotspot among the plurality of hotspots comprises a position of the at least one 360-degree camera.

### Claim 6 (depends on 4)

The method of claim 4 further comprising overlaying an asset at the new hotspot and in front of an image of the at least one object.

### Claim 7 (depends on 6)

The method of claim 6 further comprising:
if the user interacts with the asset, fading out the 360-degree video and replacing the 360-degree video with the low fidelity background video, which is associated with the at least one object;
simultaneously constructing the 3D model of the at least one object in the low fidelity background video;
fading into a position that the at least one object had occupied in the 360-degree video; and
allowing the user to interact with the 3D model of the at least one object, including allowing the user to freely move within the each segment and around the 3D model.

### Claim 8 (depends on 4)

The method of claim 4 further comprising:
after the user has completed an interaction with the 3D model, returning the user to an original hotspot among the hotspots from which the user started when interacting with the 3D model;
allowing the user to interact with a graphically displayed button at the original position that takes the user back to the 360-degree video;
fading out the low fidelity background video and the 3D model from the scene; and
deleting the 3D model and fading a high-fidelity 360-degree video into a view of the user.

### Claim 9 (depends on 1)

The method of claim 1 wherein the segments comprise equal-sized segments or non-equally sized segments.

### Claim 10 (depends on 9)

The method of claim 9 further comprising allowing the non-equally sized segments to be stitched together.

### Claim 11

A system of authoring a three-dimensional (3D) presentation, comprising:
at least one processor; and
a non-transitory computer-usable medium embodying computer program code, the computer-usable medium capable of communicating with the at least one processor, the computer program code comprising instructions executable by the at least one processor and configured for:
dividing a scene into a grid of segments, wherein at least one 360-degree camera records the scene from a center of each segment among the segments, wherein a size of the each segment is proportional to a complexity of an environment captured in the scene;
creating at least one object within the scene for a user interaction with a 3D presentation of the scene in a context of a video stream; and
using a logical center of the at least one object as a center point for recording a 360-degree video by the at least one 360-degree camera for use as a low fidelity background video when the user interacts with the 3D presentation of the scene, wherein the 3D presentation is based on a 3D model.

### Claim 12 (depends on 11)

The system of claim 11 wherein the scene comprises a large-scale scene.

### Claim 13 (depends on 11)

The method of claim 11 wherein the at least one 360-degree video comprises at least one visual cue.

### Claim 14 (depends on 11)

The system of claim 11 wherein the instructions are further configured for navigating between a plurality of hotspots in the 3D presentation using a teleporting method that translates a position of a user from a current hotspot among the plurality of hotspots and provides a motion blur as the video stream is transitioned to a new hotspot among the plurality of hotspots, which becomes a new position of the user.

### Claim 15 (depends on 4)

The system of claim 4 wherein each hotspot among the plurality of hotspots comprises a position of the at least one 360-degree camera.

### Claim 16 (depends on 4)

The system of claim 4 wherein the instructions are further configured for overlaying an asset at the new hotspot and in front of an image of the at least one object.

### Claim 17 (depends on 16)

The system of claim 16 wherein the instructions are further configured for:
if the user interacts with the asset, fading out the 360-degree video and replacing the 360-degree video with the low fidelity background video, which is associated with the at least one object;
simultaneously constructing the 3D model of the at least one object in the low fidelity background video;
fading into a position that the at least one object had occupied in the 360-degree video; and
allowing the user to interact with the 3D model of the at least one object, including allowing the user to freely move within the each segment and around the 3D model.

### Claim 18 (depends on 14)

The system of claim 14 wherein the instructions are further configured for:
after the user has completed an interaction with the 3D model, returning the user to an original hotspot among the hotspots from which the user started when interacting with the 3D model;
allowing the user to interact with a graphically displayed button at the original position that takes the user back to the 360-degree video;
fading out the low fidelity background video and the 3D model from the scene; and
deleting the 3D model and fading a high-fidelity 360-degree video into a view of the user.

### Claim 19 (depends on 11)

The system of claim 11 wherein the segments comprise equal-sized segments or non-equally sized segments.

### Claim 20

A system for authoring a three-dimensional (3D) presentation, comprising:
at least one 360-degree camera that records a scene that is divided into a grid of equal-sized segments, wherein the at least one 360-degree camera records the scene from a center of each segment among the equal-sized segments, and a size of the each segment is proportional to a complexity of an environment captured in the scene;
a 3D presentation, wherein at least one object is created within the scene for a user interaction with the 3D presentation of the scene in a context of a video stream; and
wherein a logical center of the at least one object is used as a center point for recording a 360-degree video by the at least one 360-degree camera for use as a low fidelity background video when the user interacts with the 3D presentation of the scene, wherein the 3D presentation is based on a 3D model.
