Back to results
US 2020/0090300 A1Published application
Creating shockwaves in three-dimensional depth videos and images
AssigneeSnap Inc.
InventorsSagi Katz, Eyal Zak
Life of this patent
Estimating the term…
Abstract
A virtual shockwave creation system comprises an eyewear device that includes a frame, a temple connected to a lateral side of the frame, and a depth-capturing camera. Execution of programming by a processor configures the virtual shockwave creation system to generate, for each of multiple initial depth images, a respective shockwave image by applying a transformation function to the initial three-dimensional coordinates. The virtual shockwave creation system creates a warped shockwave video including a sequence of the generated warped shockwave images. The virtual shockwave creation system presents, via an image display, the warped shockwave video.
Claims18
A virtual shockwave creation system comprising:
- an eyewear device including:
- a frame;
- a temple connected to a lateral side of the frame;
- and
- a depth-capturing camera supported by at least one of the frame or the temple, where the depth-capturing camera includes:
- (i) at least two visible light cameras with overlapping fields of view;
- or (ii) a least one visible light camera and a depth sensor;
- an image display for presenting an initial video including initial images, wherein the initial images are raw images or processed raw images that are two-dimensional;
- an image display driver coupled to the image display to control the image display to present the initial video;
- a user input device to receive a shockwave effect selection from a user to apply shockwaves to the presented initial video;
- a memory;
- a processor coupled to the depth-capturing camera, the image display driver, the user input device, and the memory;
- and
- programming in the memory, wherein execution of the programming by the processor configures the virtual shockwave creation system to perform functions, including functions to:
- present, via the image display, the initial video;
- receive, via the user input device, the shockwave effect selection from the user to apply shockwaves to the presented initial video;
- generate, via the depth-capturing camera, a sequence of initial depth images from respective initial images in the initial video, wherein:
- each of the initial depth images is associated with a time coordinate on a time (T) axis for a presentation time based on the respective initial images in the initial video;
- each of the initial depth images is formed of a matrix of vertices, each vertex representing a sampled 3D location in a respective three-dimensional scene;
- each vertex has a position attribute;
- and
- the position attribute of each vertex is based on a three-dimensional location coordinate system and includes an X location coordinate on an X axis for horizontal position, a Y location coordinate on a Y axis for vertical position, and a Z location coordinate on a Z axis for a depth position;
- in response to receiving the shockwave effect selection, based on, at least, the associated time coordinate of each of the initial depth images, generate, for each of the initial depth images, a respective warped shockwave image by applying a transformation function to vertices of the respective initial depth image based on, at least the Y and Z location coordinates and the associated time coordinate;
- create, a warped shockwave video including the sequence of the generated warped shockwave images;
- and
- present, via the image display, the warped shockwave video.ransforms a respective shockwave region of vertices grouped together along the Z axis based on, at least, the associated time coordinate of a respective initial depth image;
- and
- the transformation function moves a respective Y location coordinate of vertices in the respective shockwave region of vertices vertically upwards or downwards on the Y axis.
25 more elements
The virtual shockwave creation system of claim 3, wherein:
- the transformation function for each initial depth pixel moves the respective Y location coordinate of each vertex in the respective shockwave region of vertices vertically upwards or downwards on the Y axis to vertically fluctuate or oscillate the respective shockwave region of vertices;
- and
- the function of generating, for each of the initial depth images, the respective shockwave depth image by applying the respective transformation function to the respective initial depth image vertically fluctuates or oscillates the respective shockwave region of vertices and stores the respective initial depth image with the vertical fluctuations or oscillations as the respective shockwave depth image.
The virtual shockwave creation system of claim 6, wherein:
- applying the transformation function creates a new modified set of vertices or a three-dimensional image without a depth map.
The virtual shockwave creation system of claim 8, wherein:
- the initial video further includes:
- a later initial depth image associated with a later time coordinate on the time (T) axis for a later presentation time after the intermediate presentation time of the intermediate initial depth image in the initial video;
- and
- the function of transforming the respective shockwave region of vertices along the Z axis based on, at least, the associated time coordinate of the respective initial depth image further includes:
- transforming a far range shockwave region of vertices with farther depth positions grouped contiguously together along the Z axis for the later initial depth image based on the later time coordinate;
- and
- the far range shockwave region of vertices is farther in depth along the Z axis than the intermediate range shockwave region of vertices.
4 more elements
The virtual shockwave creation system of claim 10, wherein:
- the influence weight is greater as a height of the vertex relative to a floor plane of the respective initial depth image decreases such that the transformation function moves the Y location coordinate of the vertex vertically upwards on the Y axis to a greater extent;
- and
- the influence weight is lower as the height of the vertex relative to the floor plane increases such that the transformation function moves the Y location coordinate of the vertex vertically upwards on the Y axis to a lesser extent.
The virtual shockwave creation system of claim 12, wherein:
- the function of tracking, via the inertial measurement unit, the head orientation of the head of the wearer includes measuring, via the inertial measurement unit, the head orientation on the X axis, the Y axis, the Z axis, or the combination thereof;
- and
- in response to measuring the head orientation, determining a deviation angle of the depth-capturing camera on the X axis, the Y axis, the Z axis, or the combination thereof;
- and
- re-orienting the vertices based on the deviation angle such that one axis, either the X axis, the Y axis, or the Z axis is perpendicular to the ground.
2 more elements
The virtual shockwave creation system of claim 15, wherein:
- the host computer is a mobile device;
- the network is a wireless short-range network or a wireless local area network;
- and
- the user input device includes a touch screen or a computer mouse.
1 more element
A method comprising steps of:
- generating, via a depth-capturing camera, a sequence of initial depth images from the initial images of an initial video;
- in response to receiving a shockwave effect selection, determining, for each of the initial depth images, a respective rotation matrix to adjust, at least X and Y location coordinates of the vertices based on detected rotation of the depth-capturing camera;
- generating, for each of the initial depth images, a respective warped shockwave image by applying the respective rotation matrix and a transformation function to vertices of a respective initial depth image;
- creating, a warped shockwave video including the sequence of the generated warped shockwave images;
- and
- presenting, via an image display, the warped shockwave video.
3 more elements
The method of claim 17, wherein:
- the transformation function transforms a respective shockwave region of vertices grouped together along a Z axis based on, at least, an associated time coordinate of the respective initial depth image;
- and
- the transformation function moves a respective Y location coordinate of vertices in the respective shockwave region of vertices vertically upwards or downwards on the Y axis.
The method of claim 17, wherein:
- the function of presenting via the image display, the warped shockwave video including the sequence of the generated warped shockwave images presents an appearance of a rolling wave radially from the depth-capturing camera, radially from an object emitting a shockwave, or along a Z axis of the warped shockwave images of the warped shockwave video.
Dependencies are inferred from the claim text.