# US 2024/0005609 A1 · Efficient capture and delivery of walkable and interactive virtual reality or 360 degree video

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005609 A1
- Document ID: US20240005609A1
- Status: Published application
- Kind: A1
- Application number: 18/467,908
- Filed: 2023-09-15
- Published: 2024-01-04
- Earliest claimed: 2020-05-21
- Assignees: Virtuix Holdings Inc.
- Applicants: Virtuix Holdings Inc.
- Inventors: Jan GOETGELUK; Benjamin John DRAKES
- Web page: https://patentgrep.com/patent/US20240005609A1

## CPC

- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06T 19/003: {Navigation within 3D models or images}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06T 15/205: {Image-based rendering}
- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G06F 2203/012: Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)

## Abstract

Disclosed are systems and methods for generating a walkable 360-degree video or virtual reality (VR) environment. 360-degree video data is obtained for a real-world environment and comprises a plurality of chronologically ordered frames captured by traversing a first path through the real-world environment. One or more processing operations are applied to generate a processed 360-degree video, which can be displayed to a user of an omnidirectional treadmill. Locomotion information is received from one or more sensors of the omnidirectional treadmill, wherein the locomotion information is generated based on a physical movement on or within the omnidirectional treadmill. Using the received locomotion information, one or more playback commands for controlling playback of the processed 360-degree video are generated. One or more selected frames of the processed 360-degree video are rendered for presentation and display to the user, based on the one or more playback commands.

## Claims

### Claim 1

A method comprising:
receiving locomotion information from one or more sensors, wherein the locomotion information is generated based on physical movement of a user within an omnidirectional treadmill;
generating, using the received locomotion information, one or more playback commands for controlling playback of one or more of a plurality of processed 360-degree videos, wherein the plurality of processed 360-degree videos are generated from 360-degree video data obtained by traversing one or more of a plurality of paths through a real-world environment, the plurality of processed 360-degree videos including a plurality of virtual paths corresponding with the plurality of paths through the real-world environment and one or more junction points corresponding to locations where virtual paths of the plurality of virtual paths intersect; and
rendering one or more frames of the plurality of processed 360-degree videos based on the one or more playback commands.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
generating, based on the plurality of processed 360-degree videos, a walkable virtual reality (VR) environment including the plurality of virtual paths.

### Claim 3 (depends on 1)

The method of claim 1, wherein the received locomotion information includes a direction of locomotion on the omnidirectional treadmill.

### Claim 4 (depends on 1)

The method of claim 1, wherein the received locomotion information includes a speed of locomotion.

### Claim 5 (depends on 4)

The method of claim 4, wherein an elapsed time between rendering the one or more frames is proportional to the speed of locomotion.

### Claim 6 (depends on 1)

The method of claim 1, wherein the one or more playback commands includes adjustments to a playback speed of the plurality of processed 360-degree videos.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
calculating a direction of virtual movement along a virtual path, the calculating based on the received locomotion information; and
obtaining the one or more frames from the plurality of processed 360-degree videos in response to a determination that the calculated direction of the virtual movement corresponds to a traversal direction in which the 360-degree video data was captured along a path of the plurality of paths through the real-world environment.

### Claim 8 (depends on 1)

The method of claim 1, wherein processing the 360-degree video data to generate the processed 360-degree videos comprises performing one or more of:
stabilization;
top and tailing;
cropping;
adjusting playback speed; and
mirroring.

### Claim 9

A system comprising:
at least one processor; and
a memory having instructions therein, which when executed by the at least one processor cause the system to:
receive locomotion information from one or more sensors, wherein the locomotion information is generated based on physical movement of a user within an omnidirectional treadmill;
generate, using the received locomotion information, one or more playback commands for controlling playback of one or more of a plurality of processed 360-degree videos, wherein the plurality of processed 360-degree videos are generated from 360-degree video data obtained by traversing one or more of a plurality of paths through a real-world environment, the plurality of processed 360-degree videos including a plurality of virtual paths corresponding with the plurality of paths through the real-world environment and one or more junction points corresponding to locations where virtual paths of the plurality of virtual paths intersect; and
render one or more frames of the plurality of processed 360-degree videos based on the one or more playback commands.

### Claim 10 (depends on 9)

The system of claim 9 further comprising instructions, which when executed by the at least one processor, causes the system to:
generate, based on the plurality of processed 360-degree videos, a walkable virtual reality (VR) environment including the plurality of virtual paths.

### Claim 11 (depends on 9)

The system of claim 9, wherein the received locomotion information includes a direction of locomotion on the omnidirectional treadmill.

### Claim 12 (depends on 9)

The system of claim 9, wherein the received locomotion information includes a speed of locomotion.

### Claim 13 (depends on 12)

The system of claim 12, wherein an elapsed time between rendering the one or more frames is proportional to the speed of locomotion.

### Claim 14 (depends on 9)

The system of claim 9, wherein the one or more playback commands includes adjustments to a playback speed of the plurality of processed 360-degree videos.

### Claim 15 (depends on 9)

The system of claim 9, further comprising instructions, which when executed by the at least one processor, causes the system to:
calculate a direction of virtual movement along a first virtual path, the calculating based on the received locomotion information; and
obtain the one or more frames from the plurality of processed 360-degree videos in response to a determination that the calculated direction of the virtual movement corresponds to a traversal direction in which the 360-degree video data was captured along a path of the plurality of paths through the real-world environment.

### Claim 16 (depends on 9)

The system of claim 9, wherein processing the 360-degree video data to generate the processed 360-degree video data comprises performing one or more of:
stabilization;
top and tailing;
cropping;
adjusting playback speed; and
mirroring.

### Claim 17

At least one non-transitory computer-readable medium storing instructions therein, which when executed by at least one processor cause the at least one processor to:
receive locomotion information from one or more sensors, wherein the locomotion information is generated based on physical movement of a user within an omnidirectional treadmill;
generate, using the received locomotion information, one or more playback commands for controlling playback of one or more of a plurality of processed 360-degree videos, wherein the plurality of processed 360-degree videos are generated from 360-degree video data obtained by traversing one or more of a plurality of paths through a real-world environment, the plurality of processed 360-degree videos including a plurality of virtual paths corresponding with the plurality of paths through the real-world environment and one or more junction points corresponding to locations where virtual paths of the plurality of virtual paths intersect; and
render one or more frames of the plurality of processed 360-degree videos based on the one or more playback commands.

### Claim 18 (depends on 17)

The at least one non-transitory computer-readable medium of claim 17, further comprising instructions, which when executed by the at least one processor, causes the at least one processor to:
generate, based on the plurality of processed 360-degree videos, a walkable virtual reality (VR) environment including the plurality of virtual paths.

### Claim 19 (depends on 17)

The at least one non-transitory computer-readable medium of claim 17, wherein the received locomotion information includes a direction of locomotion on the omnidirectional treadmill.

### Claim 20 (depends on 17)

The at least one non-transitory computer-readable medium of claim 17, wherein the received locomotion information includes a speed of locomotion.
