Method of device tracking, terminal device, and storage medium

A terminal device, comprising:

• a first camera, arranged to capture a first image including a marker;
• a second camera, arranged to capture a second image including a target scene;
• an inertial measurement unit (IMU), arranged to acquire position and pose of the IMU relative to the marker;
• a non-transitory memory, arranged to store one or more computer programs;
• and
• one or more processors, arranged to execute the one or more computer programs to perform operations of:
• acquiring first information based on a first image including a marker captured by the first camera, wherein the first information comprises relative position and pose between the first camera and the marker;
• acquiring second information based on a second image including a target scene captured by the second camera, wherein the second information comprises position and pose of the second camera in the target scene, wherein the marker and the terminal device are located in the target scene;
• and
• updating the position and pose of the IMU relative to the marker based on at least one of the first information and the second information to determine position and pose of the terminal device relative to the marker, comprising:
• determining predicted information at various time points via the IMU, wherein the predicted information is the position and pose of the IMU relative to the marker at the various time points calculated by the terminal device;
• and
• taking the predicted information of a present time point of the IMU as the position and pose of the terminal device relative to the marker;
• wherein
• under a condition of the first information of a first time point being acquired, acquiring a first physical relation between the first camera and the IMU;
• determining the position and pose of the IMU relative to the marker based on the first information of the first time point and the first physical relation;
• and
• updating the predicted information of the first time point, based on the position and pose of the IMU relative to the marker, to determine the first predicted information;
• re-calculating the relative position and pose between the first camera and the marker based on the first physical relation and the first predicted information to obtain first data;
• obtaining an error between the first information of the first time point and the first data;
• updating the first physical relation based on the error;
• and
• re-calculating the predicted information of other time points after the first time point via the IMU based on the first predicted information, such that updating the position and pose of the terminal device relative to the marker.

A non-transitory computer storage medium, having a method of device tracking stored in, wherein the method is capable of being executed by a system, the system at least comprises a processor, a first camera, a second camera, and an inertial measurement unit (IMU), and the method comprises operations of:

• acquiring first information based on a first image including a marker captured by the first camera, wherein the first information comprises relative position and pose between the first camera and the marker;
• acquiring second information based on a second image including a target scene captured by the second camera, wherein the second information comprises position and pose of the second camera in the target scene, wherein the marker and the terminal device are located in the target scene;
• determining predicted information at various time points via the IMU, wherein the predicted information is the position and pose of the IMU relative to the marker at the various time points calculated by the terminal device;
• and
• taking the predicted information of a present time point of the IMU as the position and pose of the terminal device relative to the marker;
• wherein
• under a condition of the first information of a first time point being acquired, acquiring a first physical relation between the first camera and the IMU;
• determining the position and pose of the IMU relative to the marker based on the first information of the first time point and the first physical relation;
• updating the predicted information of the first time point, based on the position and pose of the IMU relative to the marker, to determine the first predicted information;
• and re-calculating the predicted information of other time points after the first time point via the IMU based on the first predicted information, such that updating the position and pose of the terminal device relative to the marker;
• and
• under a condition of the second information of a second time point being acquired, acquiring a second physical relation between the second camera and the IMU;
• performing coordinate transformation of the second information of the second time point, based on the first physical relation and the second physical relation, to determine intermediate information, wherein the intermediate information is the position and pose of the terminal device relative to the marker;
• updating the predicted information of the second time point based on the intermediate information to determine the second predicted information;
• and re-calculating the predicted information of other time points after the second time point via the IMU based on the second predicted information, such that updating the position and pose of the terminal device relative to the marker.