Method and device for real-time tracking of the pose of a camera by automatic management of a key-image database

A device for real-time tracking of the placement of a camera with respect to a 3D model, said 3D model comprising a keyframe base and a base of placements which are associated with each keyframe, each keyframe being in particular characterized by reference points, said device comprising a first tracking module comprising:

• a sub-module for receiving a video stream from the camera composed of a plurality of images,
• a sub-module for mapping reference points of the last received image of the video stream, referred to as the current image, with reference points of a keyframe of the 3D model by a mapping algorithm,
• a sub-module for estimating the current placement of the camera from the keyframe, referred to as the selected keyframe, comprising the highest number of reference points in common with the current image, by estimating the shift between the placement of the camera associated with the selected keyframe from the placement base and the current placement of the camera associated with the current image,
• characterized in that the device further comprises a module for managing the keyframe base comprising:
• a sub-module comprising a loss-of-placement counter incremented if the number of reference points in common between the selected keyframe and the current image is lower than or equal to a predetermined threshold,
• a sub-module for transferring the current image of the video stream to a buffer memory of the device if the number of reference points in common between the selected keyframe and the current image is higher than the predetermined threshold,
• a sub-module for comparing the value of the loss-of-placement counter with a predetermined value,
• a sub-module for analyzing the quality of the estimation of the placement of the camera from the image stored in the buffer memory,
• a sub-module for analyzing the sharpness of the image stored in the buffer memory from data representative of the movement of the camera,
• a sub-module for managing the buffer memory, configured to add the image stored in the keyframe base if the quality of the estimation of the placement of the camera from the stored image and the sharpness of the stored image conform to predetermined criteria, and to remove the image stored in the buffer memory otherwise.

A computer program product for real-time tracking of the placement of a camera with respect to a 3D model, said 3D model comprising a keyframe base and a base of placements which are associated with each keyframe, each keyframe being in particular characterized by reference points, said computer program product comprising program code instructions which when executing, perform steps of a a first thread of execution comprising:

• a step of receiving a video stream from the camera composed of a plurality of images,
• a step of mapping reference points of the last received image of the video stream, referred to as the current image, with reference points of a keyframe of the 3D model by a mapping algorithm,
• a step of estimating the current placement of the camera from the keyframe, referred to as the selected keyframe, comprising the highest number of reference points in common with the current image, by estimating the shift between the placement of the camera associated with the selected keyframe from the placement base and the current placement of the camera associated with the current image,
• a step of incrementing a loss-of-placement counter on condition that the number of reference points in common between the selected keyframe and the current image is lower than or equal to a predetermined threshold,
• a step of storing the current image of the video stream in a buffer memory on condition that the number of reference points in common between the selected keyframe and the current image is higher than the predetermined threshold, and
• on condition that the loss-of-placement counter is higher than or equal to a predetermined value performing steps of a second thread of execution comprising:
• a step of analyzing the quality of the estimation of the placement of the camera from the image stored in the buffer memory,
• a step of analyzing the sharpness of the image stored in the buffer memory from data representative of the movement of the camera,
• a step of adding the image stored in the buffer memory to the keyframe base as a new keyframe on condition that the quality of the estimation of the placement of the camera from the stored image and the sharpness of the stored image conform to predetermined criteria, but otherwise performing a step of removing the image stored in the buffer memory.