Determining 6D pose estimates for augmented reality (AR) sessions
AssigneeStreem, LLC
InventorsFlora Ponjou Tasse, Pavan K. Kamaraju, Ghislain Fouodji Tasse, Sean M. Adkinson
Life of this patent
Estimating the term…
Abstract
Embodiments include systems and methods for determining a 6D pose estimate associated with an image of a physical 3D object captured in a video stream. An initial 6D pose estimate is inferred and then further iteratively refined. The video stream may be frozen to allow the user to tap or touch a display to indicate a location of the user-input keypoints. The resulting 6D pose estimate is used to assist in replacing or superimposing the physical 3D object with digital or virtual content in an augmented reality (AR) frame.
Claims19
A method for determining a calculated six degrees of freedom (6D) pose estimate associated with an image of a physical three-dimensional (3D) object captured in a video stream, comprising:
- receiving, from a user, a location of user-input keypoints for the physical 3D object as displayed in a video frame of the video stream, wherein the video stream is frozen to allow the user to input the user-input keypoints;
- inferring an initial 6D pose estimate;
- obtaining inferred keypoints by projecting the user-input keypoints using the initial 6D pose estimate;
- and
- generating, using at least the location of the user-input keypoints and the inferred keypoints, the calculated 6D pose estimate to assist in replacing or superimposing the physical 3D object with digital or virtual content in an augmented reality (AR) frame.
A non-transitory computer readable medium (CRM) comprising instructions that, when executed by an apparatus, cause the apparatus to determine a calculated six degrees of freedom (6D) pose estimate of a physical three-dimensional (3D) object captured in a video stream, wherein the instructions include to:
- receive, from a user, a location of user-input keypoints for the physical 3D object in a video frame, wherein the video stream is comprised of a plurality of video frames each including image data and the video stream is frozen to allow the user to indicate the location of the user-input keypoints in an image of the video frame;
- infer an initial 6D pose estimate;
- obtain inferred keypoints by projecting the user-input keypoints using the initial 6D pose estimate;
- and
- generate, using at least the location of the user-input keypoints and the inferred keypoints, the calculated 6D pose estimate to assist in placing digital or virtual content in a similar location in the video frame as the physical 3D object.
An apparatus, comprising:
- a memory;
- and
- a processor coupled to the memory to receive instructions from the memory to cause the apparatus to:
- display a video frame of a video stream including an image of a physical three-dimensional (3D) object on a touchscreen of the apparatus;
- prompt a user to input locations of a plurality of user-input keypoints of the physical 3D object on the touchscreen, wherein to input locations of the physical 3D object includes to input, by the user, the locations of the plurality of the user-input keypoints;
- infer an initial six degrees of freedom (6D) pose estimate;
- obtain inferred keypoints by projecting the user-input keypoints using the initial 6D pose estimate;
- and
- generate, using at least the locations of the plurality of the user-input keypoints and the inferred keypoints, a calculated 6D pose estimate to assist in placing digital or virtual content in a similar location in an augmented reality (AR) frame.
Dependencies are inferred from the claim text.