Depth of field in video based on gaze
AssigneeApple Inc.
InventorsAlexandre Da Veiga, Timothy T. Chong
Life of this patent
Estimating the term…
Abstract
Various implementations disclosed herein include devices, systems, and methods that modify a video to enable replay of the video with a depth-based effect based on gaze of a user during capture or playback of the video. For example, an example process may include tracking a gaze direction during a capture or a playback of a video, wherein the video includes a sequence of frames including depictions of a three-dimensional (3D) environment, identifying a portion of the 3D environment depicted in the video based on the gaze direction, determining a depth of the identified portion of the 3D environment, and modifying the video to enable replay of the video with a depth-based effect based on the depth of the identified portion.
Claims16
A method comprising:
- at an electronic device having a processor:
- tracking a gaze direction during a capture or a playback of a video, wherein the video comprises a sequence of frames comprising depictions of a three-dimensional (3D) environment, and wherein the video is a stereoscopic video, each frame comprising a pair of images;
- identifying a portion of the 3D environment depicted in the video based on the gaze direction;
- determining one or more depths of the identified portion of the 3D environment;
- modifying the video to enable replay of the video with a depth-based effect based on the one or more depths of the identified portion, wherein modifying the video comprises storing at least one of an identification of the identified portion and the one or more depths of the identified portion in association with the video;
- and
- exporting the modified video as a monoscopic video having a single image for each frame.
A device comprising:
- a non-transitory computer-readable storage medium;
- and
- one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising:
- tracking a gaze direction during a capture or a playback of a video, wherein the video comprises a sequence of frames comprising depictions of a three-dimensional (3D) environment, and wherein the video is a stereoscopic video, each frame comprising a pair of images;
- identifying a portion of the 3D environment depicted in the video based on the gaze direction;
- determining one or more depths of the identified portion of the 3D environment;
- modifying the video to enable replay of the video with a depth-based effect based on the one or more depths of the identified portion, wherein modifying the video comprises storing at least one of an identification of the identified portion and the one or more depths of the identified portion in association with the video;
- and
- exporting the modified video as a monoscopic video having a single image for each frame.
A non-transitory computer-readable storage medium, storing program instructions executable by one or more processors of an electronic device to perform operations comprising:
- tracking a gaze direction during a capture or a playback of a video, wherein the video comprises a sequence of frames comprising depictions of a three-dimensional (3D) environment, and wherein the video is a stereoscopic video, each frame comprising a pair of images;
- identifying a portion of the 3D environment depicted in the video based on the gaze direction;
- determining one or more depths of the identified portion of the 3D environment;
- modifying the video to enable replay of the video with a depth-based effect based on the one or more depths of the identified portion, wherein modifying the video comprises storing at least one of an identification of the identified portion and the one or more depths of the identified portion in association with the video;
- and
- exporting the modified video as a monoscopic video having a single image for each frame.
Dependencies are inferred from the claim text.