Method for estimating the pose of a camera in the frame of reference of a three-dimensional scene, device, augmented reality system and computer program therefor

An apparatus comprising:

• a device for estimating a pose of a camera in a reference frame of a three-dimensional scene, said device comprising:
• a computing machine dedicated to or configured to:
• obtaining a color intensity image of the scene captured by the camera, called a current image;
• extracting a plurality of points of interest from the current image, each point being invariant by geometric transformation of the image;
• forming a plurality of patches in the image of color intensities, each patch comprising a point of interest of the extracted plurality;
• predicting 3D locations of the points of interest of the plurality of patches in a scene repository by applying an automatic prediction system comprising a decision random forest, comprising decision trees, each decision tree comprising separator nodes, each node being capable of binary separating a group of feature vectors from an upstream node into subgroups, transmitting the subgroups to downstream separator nodes and leaf nodes, capable of storing the 3D locations of the patches associated with a subgroup received from an upstream separator node, said system having been trained using a training set comprising patches from a plurality of images of the scene acquired by the camera from a plurality of viewpoints, each patch from the plurality of images being associated with a 2D position of its point of interest in an image repository and a 3D position of its point of interest in the scene repository, the prediction further comprising an extraction of a feature vector per patch and the application of the automatic prediction system predicts a plurality of 3D locations per extracted feature vector;
• and
• estimating the pose of a camera for the current image, by mapping the 2D positions of the plurality of points of interest and reprojections in the current image frame of reference of the predicted 3D locations,
• and wherein the computing machine is configured to implement in a preliminary phase a training of the prediction system comprising a binary separation test based on a calculation of distance between the feature vector extracted from a patch of the training set and a vector of reference features and a comparison of the calculated distance with respect to a predetermined threshold.

A non-transitory computer-readable recording medium on which is recorded a computer program comprising program code instructions for implementing a method for estimating a pose of a camera in a frame of reference of a three-dimensional scene, when the instructions are executed by a processor, wherein the instructions configure the processor to:

• obtain an image of colour intensities of the scene captured by the camera, called a current image;
• extract a plurality of points of interest from the current image, each point being invariant by geometric transformation of the image;
• form a plurality of patches in the image of colour intensities, each patch comprising a point of interest of the extracted plurality;
• predict the 3D locations of the points of interest of the plurality of patches in a frame of reference of the scene, by application of an automatic prediction system comprising a decision random forest, comprising decision trees, each decision tree comprising separator nodes, each node being capable of binary separating a group of feature vectors from an upstream node into subgroups, transmitting the subgroups to downstream separator nodes and leaf nodes, capable of storing the 3D locations of the patches associated with a subgroup received from an upstream separator node, said system having been trained with aid of a training set comprising patches from a plurality of images of the scene acquired by the camera from a plurality of points of view, each patch from the plurality of images being associated in said set with a 2D position of its point of interest in a frame of reference of the image and with a 3D position of its point of interest in the frame of reference of the scene, the training of the prediction system in a preliminary phase comprising a binary separation test based on a calculation of distance between the feature vector extracted from a patch of the training set and a vector of reference features and a comparison of the calculated distance with respect to a predetermined threshold, the prediction further comprising an extraction of a feature vector per patch and the application of the automatic prediction system predicts a plurality of 3D locations per extracted feature vector;
• and
• estimate a camera pose for the current image, by mapping the 2D positions of the plurality of points of interest and reprojections in the current image frame of reference of the predicted 3D locations.