Image-based localization
AssigneeMicrosoft Technology Licensing, LLC
InventorsJohannes Lutz Schonberger, Marc Andre Leon Pollefeys
Life of this patent
Estimating the term…
Abstract
A method for image-based localization includes, at a camera device, capturing a plurality of images of a real-world environment. A first set of image features are detected in a first image of the plurality of images. Before additional sets of image features are detected in other images of the plurality, the first set of image features is transmitted to a remote device configured to estimate a pose of the camera device based on image features detected in the plurality of images. As the additional sets of image features are detected in the other images of the plurality, the additional sets of image features are transmitted to the remote device. An estimated pose of the camera device is received from the remote device.
Claims18
A method for image-based localization, comprising:
- at a camera device, capturing a plurality of images of a real-world environment;
- detecting a first set of image features in a first image of the plurality of images;
- before additional sets of image features are detected in additional images of the plurality of images, transmitting the first set of image features to a remote device configured to estimate a pose of the camera device;
- receiving, from the remote device, a first estimated pose of the camera device, the first estimated pose having been estimated based on the first set of image features detected in the first image of the plurality of images;
- as the additional sets of image features are detected in the additional images of the plurality of images, transmitting the additional sets of image features to the remote device;
- and
- receiving, from the remote device, a second estimated pose of the camera device, the second estimated pose having been estimated based on multiple sets of image features detected in two or more images of the plurality of images.
A camera device, comprising:
- one or more cameras;
- and
- a logic machine, configured to:
- capture a plurality of images of a real-world environment via the one or more cameras;
- detect a first set of image features in a first image of the plurality of images;
- before additional sets of image features are detected in additional images of the plurality of images, transmit the first set of image features to a remote device configured to estimate a pose of the camera device;
- receive, from the remote device, a preliminary estimated pose of the camera device, the preliminary estimated pose having been estimated based on the first set of image features detected in the first image of the plurality of images, the preliminary estimated pose received before the additional sets of image features are detected for every image of the plurality of images, and the preliminary estimated pose having an associated confidence value that is proportional to a quantity of image features transmitted to the remote device before the preliminary estimated pose was estimated;
- as the additional sets of image features are detected in the additional images of the plurality of images, transmit the additional sets of image features to the remote device;
- and
- receive, from the remote device, a second estimated pose of the camera device, the second estimated pose having been estimated based on more detected image features than the preliminary estimated pose, and the second estimated pose having a correspondingly higher confidence value.
A computing device, comprising:
- a logic machine configured to:
- receive, from a camera device, a plurality of images of a real-world environment of the camera device captured by one or more cameras of the camera device;
- as the plurality of images of the real-world environment are received, detect sets of image features in each image of the plurality of images;
- as the sets of image features are detected, identify correspondences between detected image features and map features of a digital environment map corresponding to the real-world environment;
- as the correspondences are identified, progressively output a plurality of estimated poses of the camera device based on the identified correspondences, where each estimated pose has a corresponding confidence value proportional to a quantity of the correspondences identified when each respective estimated pose was output;
- upon estimating a preliminary pose of the camera device having a corresponding confidence value that exceeds a preliminary confidence threshold, and before detecting the sets of image features for every image of the plurality of images, report the first pose to the camera device;
- and
- upon estimating a second pose of the camera device having a corresponding confidence value that exceeds a final confidence threshold higher than the preliminary confidence threshold, report the second pose to the camera device.
Dependencies are inferred from the claim text.