Back to results
US 9,852,543 B2Granted
Automated three dimensional model generation
AssigneeSnap Inc.
InventorsSamuel Edward Hare, Ebony James Charlton, Andrew James McPhee, Michael John Evans
Life of this patent
Estimating the term…
Abstract
In various example embodiments, a system and methods are presented for generation and manipulation of three dimensional (3D) models. The system and methods cause presentation of an interface frame encompassing a field of view of an image capture device. The systems and methods detect an object of interest within the interface frame, generate a movement instruction with respect to the object of interest, and detect a first change in position and a second change in position of the object of interest. The systems and methods generate a 3D model of the object of interest based on the first change in position and the second change in position.
Claims15
A method, comprising:
- causing presentation of a graphical user interface frame encompassing a graphical rendering of a field of view of an image capture device of a mobile computing device;
- detecting a face within the graphical user interface frame and the field of view of the image capture device;
- generating a movement instruction directing movement of the face within the field of view of the image capture device;
- detecting a first change in position of the face within the field of view of the image capture device by:
- identifying a set of facial tracking points on the face;
- identifying a first key frame, in which the set of facial tracking points have a set of first positions;
- determining a change in position of one or more facial tracking points by identifying a trajectory for each of the one or more facial tracking points, determining an average length of the trajectories, and determining the average length exceeds a trajectory threshold;
- responsive to determining the change in position and the average length exceeding the trajectory threshold, identifying a second key frame where the one or more facial tracking points have a second position;
- detecting a second change in position of the face within the field of view of the image capture device;
- identifying one or more subsequent key frames based on one or more changes in position of the one or more facial tracking points along the trajectories;
- based on the first key frame, the second key frame, and the one or more subsequent key frames, generating a set of relative position estimates for the mobile computing device with respect to the face, a relative position estimate of the set of relative position estimates being generated for each key frame;
- based on the first key frame, the second key frame, the one or more subsequent key frames, and the set of relative position estimates, generating a set of depth maps including a depth map for each key frame, the set of depth maps generated by:
- generating a first depth map having a first resolution, and
- based on the first depth map, generating a second depth map having a second resolution higher than the first resolution;
- and
- generating a three dimensional model of the face.
13 more elements
A system, comprising:
- one or more processors;
- an image capture device operative coupled to the one or more processors;
- and
- a non-transitory processor-readable storage medium storing processor executable instructions that, when executed by the one or more processors, causes the one or more processors to perform operations comprising:
- causing presentation of a graphical user interface frame encompassing a graphical rendering of a field of view of an image capture device of a mobile computing device;
- detecting a face within the graphical user interface frame and the field of view of the image capture device;
- generating a movement instruction directing movement of the face within the field of view of the image capture device;
- detecting a first change in position of the face within the field of view of the image capture device by:
- identifying a set of facial tracking points on the face;
- identifying a first key frame, in which the set of facial tracking points have a set of first positions;
- determining a change in position of one or more facial tracking points by identifying a trajectory for each of the one or more facial tracking points, determining an average length of the trajectories, and determining the average length exceeds a trajectory threshold;
- responsive to determining the change in position and the average length exceeding the trajectory threshold, identifying a second key frame where the one or more facial tracking points have a second position;
- detecting a second change in position of the face within the field of view of the image capture device;
- identifying one or more subsequent key frames based on one or more changes in position of the one or more facial tracking points along the trajectories;
- based on the first key frame, the second key frame, and the one or more subsequent key frames, generating a set of relative position estimates for the mobile computing device with respect to the face, a relative position estimate of the set of relative position estimates being generated for each key frame;
- based on the first key frame, the second key frame, the one or more subsequent key frames, and the set of relative position estimates, generating a set of depth maps including a depth map for each key frame, the set of depth maps generated by:
- generating a first depth map having a first resolution, and
- based on the first depth map, generating a second depth map having a second resolution higher than the first resolution;
- and
- generating a three dimensional model of the face.
17 more elements
A non-transitory processor-readable storage medium storing processor executable instructions that, when executed by one or more processors of a mobile computing device, causes the mobile computing device to perform operations comprising:
- causing presentation of a graphical user interface frame encompassing a graphical rendering of a field of view of an image capture device of a mobile computing device;
- detecting a face within the graphical user interface frame and the field of view of the image capture device;
- generating a movement instruction directing movement of the face within the field of view of the image capture device;
- detecting a first change in position of the face within the field of view of the image capture device by:
- identifying a set of facial tracking points on the face;
- identifying a first key frame, in which the set of facial tracking points have a set of first positions;
- determining a change in position of one or more facial tracking points by identifying a trajectory for each of the one or more facial tracking points, determining an average length of the trajectories, and determining the average length exceeds a trajectory threshold;
- responsive to determining the change in position and the average length exceeding the trajectory threshold, identifying a second key frame where the one or more facial tracking points have a second position;
- detecting a second change in position of the face within the field of view of the image capture device;
- identifying one or more subsequent key frames based on one or more changes in position of the one or more facial tracking points along the trajectories;
- based on the first key frame, the second key frame, and the one or more subsequent key frames, generating a set of relative position estimates for the mobile computing device with respect to the face, a relative position estimate of the set of relative position estimates being generated for each key frame;
- based on the first key frame, the second key frame, the one or more subsequent key frames, and the set of relative position estimates, generating a set of depth maps including a depth map for each key frame, the set of depth maps generated by:
- generating a first depth map having a first resolution, and
- based on the first depth map, generating a second depth map having a second resolution higher than the first resolution;
- and
- generating a three dimensional model of the face.
13 more elements
Dependencies are inferred from the claim text.