Partial passthrough in virtual reality
AssigneeMeta Platforms Technologies, LLC
InventorsMichael James LeBeau, John Nicholas Jitkoff
Life of this patent
Estimating the term…
Abstract
Systems and methods for providing partial passthrough video to a user of a virtual reality device are disclosed herein. Providing the partial passthrough video can include detecting a hand passthrough trigger event and identifying a hand passthrough video feed. Providing partial passthrough video can further include aligning the hand passthrough video feed with a virtual environment presented to a user by the virtual environment and, based on the aligning of the hand passthrough video feed with the virtual environment, overlaying the hand passthrough video feed on the virtual environment.
Claims20
A method for providing partial passthrough video to a user of a virtual reality device, the method comprising:
- displaying, to the user, a computer generated virtual environment;
- detecting a hand passthrough trigger event based on a user action and/or a condition that occurs in the user's real-world environment;
- in response to detecting the hand passthrough trigger event, identifying a hand passthrough video feed, from one or more cameras of the virtual reality device, that captures the user's real-world environment;
- aligning the hand passthrough video feed with a portion of the virtual environment by matching a location of a real-world object and/or a location of a real-world hand of the user to a corresponding location in the virtual environment;
- and
- dynamically overlaying the hand passthrough video feed, comprising the real world hand of the user and/or the real-world object, over the aligned portion of the virtual environment, wherein the dynamically overlaying provides, to the user via the hand passthrough video, a depth accurate view of a) the real-world hand of the user and/or b) the real-world object.
A computing system for providing partial passthrough video to a user of a virtual reality device, the computing system comprising:
- one or more processors;
- and one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
- displaying, to the user, a computer generated virtual environment;
- detecting a hand passthrough trigger event based on a user action and/or a condition that occurs in the user's real-world environment;
- in response to detecting the hand passthrough trigger event, identifying a hand passthrough video feed, from one or more cameras of the virtual reality device, that captures the user's real-world environment;
- aligning the hand passthrough video feed with a portion of the virtual environment by matching a location of a real-world object and/or a location of a real-world hand of the user to a corresponding location in the virtual environment;
- and
- dynamically overlaying the hand passthrough video feed, comprising the real world hand of the user and/or the real-world object, over the aligned portion of the virtual environment, wherein the dynamically overlaying provides, to the user via the hand passthrough video, a depth accurate view of a) the real-world hand of the user and/or b) the real-world object.
A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process comprising:
- displaying, to the user, a computer generated virtual environment;
- detecting a hand passthrough trigger event based on a user action and/or a condition that occurs in the user's real-world environment;
- in response to detecting the hand passthrough trigger event, identifying a hand passthrough video feed, from one or more cameras of the virtual reality device, that captures the user's real-world environment;
- aligning the hand passthrough video feed with a portion of the virtual environment presented to the user by the virtual reality device by matching a location of a real-world object and/or a location of a real-world hand of the user to a corresponding location in the virtual environment;
- and
- dynamically overlaying the hand passthrough video feed, comprising the real world hand of the user and/or the real-world object, over the aligned portion of the virtual environment, wherein the dynamically overlaying provides, to the user via the hand passthrough video, a depth accurate view of a) the real-world hand of the user and/or b) the real-world object.
Dependencies are inferred from the claim text.