Reprojecting holographic video to enhance streaming bandwidth/quality
AssigneeMicrosoft Technology Licensing, LLC
InventorForrest Power Trepte
Life of this patent
Estimating the term…
Abstract
Improved video compression and video streaming systems and methods are disclosed for environments where camera motion is common, such as cameras incorporated into head-mounted displays. This is accomplished by combining a 3D representation of the shape of the user's environment (walls, floor, ceiling, furniture, etc.), image data, and data representative of changes in the location and orientation (pose) of the camera between successive image frames, thereby reducing data bandwidth needed to send streaming video in the presence of camera motion.
Claims20
A computer system for compressing video data, comprising:
- one or more processors;
- and
- one or more computer-readable media having stored executable instructions that are operable, when executed by the one or more processors, to cause the computer system to perform at least the following:
- capture successive frames of video from a camera incorporated into a first device, the successive frames of video comprising one or more first frame image(s) and one or more second frame image(s) and including surface (depth) information;
- detect a change in a location and orientation (pose) of the camera between a first location and orientation (pose) of the one or more first frame image(s) and a second location and orientation (pose) of the one or more second frame image(s);
- generate re-projected one or more first frame image(s) as viewed from the second location and orientation (pose);
- pass the re-projected one or more first frame image(s) to a video compressor for computing differences between the re-projected one or more first frame image(s) and the one or more second frame image(s);
- and
- generate compressed video data comprising only the differences between the re-projected one or more first frame image(s) and the one or more second frame image(s).
A video compression method comprising:
- capturing successive frames of video from a camera incorporated into a first device, the successive frames of video comprising one or more first frame image(s) and one or more second frame image(s) and including surface (depth) information;
- detecting a change in a location and orientation (pose) of the camera between a first location and orientation (pose) of the one or more first frame image(s) and a second location and orientation (pose) of the one or more second frame image(s);
- generating re-projected one or more first frame image(s) as viewed from the second location and orientation (pose);
- passing the re-projected one or more first frame image(s) to a video compressor for computing differences between the re-projected one or more first frame image(s) and the one or more second frame image(s);
- and
- generating compressed video data comprising only the differences between the re-projected one or more first frame image(s) and the one or more second frame image(s).
One or more computer-readable media hardware storage device(s) having stored executable instructions that are executable by one or more processors of a computer system to cause the computer system to perform at least the following:
- capture successive frames of video from a camera incorporated into a first device, the successive frames of video comprising one or more first frame image(s) and one or more second frame image(s) and including surface (depth) information;
- detect a change in a location and orientation (pose) of the camera between a first location and orientation (pose) of the one or more first frame image(s) and a second location and orientation (pose) of the one or more second frame image(s);
- generate re-projected one or more first frame image(s) as viewed from the second location and orientation (pose);
- pass the re-projected one or more first frame image(s) to a video compressor for computing differences between the re-projected one or more first frame image(s) and the one or more second frame image(s);
- and
- generate compressed video data comprising only the differences between the re-projected one or more first frame image(s) and the one or more second frame image(s).
Dependencies are inferred from the claim text.